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Executive Summary 



To expand and improve after-school opportunities for children, New 
Jersey After 3 (NJ After 3) has sought to develop and implement a comprehensive 
system of high-quality after-school programs throughout the state. Specifically, 
NJ After 3 seeks to increase the number of after-school programs in New Jersey 
that provide a safe environment for children during after-school hours, offer 
enriching academic activities and homework assistance, and expose children to 
nurturing individuals and meaningful experiences that promote intellectual, 
physical, social, and artistic development. Under an agreement with NJ After 3, 
Policy Studies Associates, Inc., is conducting a three-year, longitudinal evaluation 
of the statewide initiative. 



Goals of the NJ After 3 Initiative 

In surveys and interviews, NJ After 3 site coordinators said that they 
served student populations characterized by diverse needs. Accordingly, local 
program goals reflected staff members’ desire to respond to as many of these 
needs as possible. Most site coordinators indicated in survey responses that their 
main goals were to provide positive adult guidance, a safe environment, and 
opportunities for social development. Other goals included: improving academic, 
health, and other life skills; providing opportunities for recreation and cultural 
enrichment; stimulating students’ interests in learning; and helping students learn 
in non-traditional ways through exposure to positive new experiences. Several 
site directors also reported a desire to raise awareness among parents of the need 
for them to support their children’s social, physical, and cognitive development. 



Participant Characteristics and Attendance Levels 

A total of 1 1,108 students in grades K-8 participated in NJ After 3- 
supported programs during the 2005-06 academic year. Boys and girls were 
evenly represented among participants, and most participants were African 
American or Latino. About half of all participants were eligible to receive free or 
reduced-price lunch. ^ Very few were English Language Learners or received 
special education services. Participants were distributed across grade levels, with 
36 percent of participants in grades K-2, 37 percent in grades 3-5, and 27 percent 
in grades 6-8. 



* The initiative’s youth database did not specify the free or reduced price lunch status of 30 
percent of NJA3 participants, so the reported percent of youth who received free or reduced price 
lunch may be an under-estimate. This is quite likely given the reluctance of many to report in the 
affirmative on this data element. 




As part of our analysis, students were categorized into one of three levels 
of participation, based on the number of days they attended the program. 

Students were identified as “highly active” if they attended a NJ After 3 program 
for at least 80 days and attended at least 80 percent of the days that they were 
enrolled in the program during the school year. An “active” participant was one 
who attended at least 60 days and at least 60 percent of the days during which 
they were enrolled. “Non-active” participants were those who attended fewer 
than 60 days or less than 60 percent of the days during which they were enrolled. 
Attendance rates were calculated based on the number of days each student 
attended their NJ After 3 program from their date of enrollment (numerator) and 
the number of days it was possible for the student to attend (denominator). In 
2005-06, 43 percent of K-8 participants met the evaluation’s criteria for highly 
active participation, and another 20 percent met the criteria for active 
participation. On average, participants attended 73 percent of the program 
sessions available to them. These averages are considered high among after- 
school programs nationally. Attendance was generally higher among younger 
students than among older students, which is consistent with national patterns. 



Overview of NJ After 3 Programming 

In 2005-06, NJ After 3 programs offered an array of academic, artistic, 
social, health, civic, and athletic activities intended to build student skills and 
maximize students’ exposure to new activities. Site coordinators reported a fairly 
consistent set of academic program offerings across the sites. All or nearly all of 
the programs offered opportunities for students to do their homework, participate 
in math and other learning games, and practice reading and writing. Similarly, 
almost every program offered arts-related and athletic activities such as visual 
arts, dance, free time for recreation, and organized team sports. More than half of 
the site coordinators said that students participated in service projects, discussed 
current events, and learned about different careers during the after-school 
program. A few sites reported that they provided opportunities for civic 
engagement, community service, and career exploration. 

The evaluation assessed the context within which program activities took 
place. Using a detailed observation guide, we observed activities and classes 
during site visits in order to assess student engagement, instruction, activity 
content, and staff and student relationships. We found significant variation in 
these measurements across activities. While opportunities for thinking 
analytically, developing relationships with staff, contributing to discussions, 
making meaningful decisions, assuming leadership roles, and collaborating with 
peers may be available within a program, these experiences were not all available 
within a single class or activity. This suggests that exposure to a variety of 
instructors and activities, each with a different focus or goal, maximizes students’ 
opportunities to learn and promotes well-rounded development. 




The staff employed by NJ After 3 were generally well qualified for their 
jobs, even though most worked for the program part-time. Almost half of all NJ 
After 3 program staff had bachelors degrees or higher, and one-quarter were 
certified teachers. Among site coordinators, nearly all had a four-year college 
degree and at least one year’s experience as an after-school program director or 
staff member. Most program employees worked fewer than 19 hours per week at 
the program, and most staff worked directly with students. 

The student-staff ratios observed during site visits indicated that the 
programs were operating within the parameters established by NJ After 3. In fact, 
at an average of six participants to every staff member, student-staff ratios were 
lower than required by NJ After 3. Most site coordinators said that student groups 
were small enough to allow staff to meet participants’ individual needs. 

Almost all of the executive directors of provider organizations reported 
that their organization delivered some type of training to site coordinators or other 
staff members. Other data suggest, however, that there was uneven access to 
professional development and training opportunities and that some individuals 
received many more hours of training than did others. When asked to identify the 
professional development that had been most valuable to their after-school staff, 
the majority of site coordinators cited classroom management, youth development 
training, and training in academic enrichment and literacy. 

In interviews, parents expressed satisfaction with program staff, often 
describing them as trustworthy, committed, capable, caring, and fair. Many 
attributed their child’s social development and improvements in confidence and 
self-esteem to the family-like atmosphere they said was fostered by program staff. 
Students also expressed satisfaction with program staff. For example, large 
majorities of participating children reported positive interactions with NJ After 3 
staff members, agreed that staff cared about them, and reported that they could 
talk to staff about things that were bothering them. Most students also reported 
that they had close friends in the program and that relationships among 
participants were good. 

Almost all of the site coordinators said that they reached out to parents and 
the community at least once a month by calling home, arranging opportunities for 
communication between parents and representatives from local agencies, and 
meeting with one or more parents. Nevertheless, parent involvement in program 
activities was limited, because work and family responsibilities made it difficult 
for parents to attend the program during after-school hours. Those who said they 
were involved often chaperoned occasional field trips or prepared and served food 
at special program events and celebrations in the evenings or on weekends. 

Community agencies typically got involved in program activities through 
the provision of resources that supported program goals. For example, they 
offered special programs and services to youth, either on- or off-site, provided 




funding through grants or contracts, and referred students in need of after-school 
service to NJ After 3 programs. 

The quality of the relationships between individual programs and schools 
varied considerably. For example, at over half of the programs that we visited, 
staff informed us that they did not have very good relationships with school-day 
teachers. Tension focused on the use of shared space and materials. 

Relationships between school-day staff and after-school staff were not poor in all 
locations. Some after-school staff told us that they frequently spoke with school- 
day staff to understand what students were doing in the classroom, so that they 
could provide targeted assistance or ensure that homework was completed. 

Several site coordinators also described efforts to align their program content with 
the school-day program. 



Baseline Information on Intended Program 
Outcomes 

The evaluation tracked early program outcome indicators in two areas, 
after-school availability and program sustainability. Parents at each of the 
locations we visited reported that the program offered through NJ After 3 was 
their only after-school option because it was free, conveniently located, and safe. 
In fact, all of the parents with whom we spoke said that there were no other 
suitable after-school options in their community that satisfied their dual needs for 
safety and affordability. Students also indicated that the NJ After 3 programming 
filled a vacuum that would not otherwise be filled with safe, positive, meaningful 
activities. 

The majority of site coordinators reported that they had been providing 
after-school services at their current location prior to receiving the grant award 
from NJ After 3, and several indicated that they had provided after-school 
programming at the site for more than five years. All said that earlier program 
services had been less extensive and served fewer students, compared to current 
programs. 

In 2005-06, NJ After 3 provided a significant proportion of the financial 
support for the programs. On average, almost three-quarters of each site’s budget 
was provided by NJ After 3. The remaining budget resources came from general 
organizational funds, funds from other state sources, fees charged to families, 
allocations from other municipal sources, federal funding sources, and other 
organizations. The evaluation found that support from NJ After 3 had enabled 
programs to expand and improve classes and activities, train staff, and increase 
capacity so that more students were served. A few sites also reported that the 
additional resources had facilitated the establishment of new partnerships with 
public schools and cultural organizations. A smaller number of sites reported that 
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the NJ After 3 resources increased their ability to leverage corporate, private, or 
municipal funds. 



Baseline Information on Intended Participant 
Outcomes 

A significant majority of student participants indicated that the NJ After 3 
program had given them a chance to do new things, work on tasks that made them 
think, and participate in activities that held their interest. At least half agreed that 
the program allowed them to do things that they usually did not get to do 
elsewhere, gave them many activities to choose from, and provided an 
opportunity for them to become involved in community service. Students in 
grades 3-5 expressed higher levels of approval than did students in grades 6-8. 

Most students also agreed with several statements designed to assess their 
perception of whether the program had helped them with their schoolwork, 
promoted academic achievement, and fostered a sense of attachment and 
belonging to the program and staff. Significant variations were evident among 
groups of students, however. For example, students who attended the program at 
the “highly active” level scored significantly higher on the academic benefits 
scale than their peers who did not attend as often. Similarly, on average, low- 
income students (those who were eligible to receive free or reduced-price 
lunches) were significantly more likely than other students to agree with 
statements about the academic benefits of the program. 

To provide baseline data that the evaluation will track over time, 
reading/language arts and homeroom teachers completed a brief report on each NJ 
After 3 participant in grades 3-6 who attended one of the 10 programs in the 
evaluation’s in-depth sample. Teachers assessed the skills and behaviors that 
after- school participants exhibited during the regular school day. According to 
responding teachers, most of the students who participated in NJ After 3 programs 
“always” or “often” demonstrated behaviors that indicated they were 
academically engaged and possessed important interpersonal, technological, and 
study skills. Most also met or exceeded the school’s grade-level expectations in 
reading and language arts skills. However, teachers indicated that fewer than half 
“always” or “often” demonstrated academic motivation (eagerness to learn, 
willingness to take on challenges, ability to stay on target, and responsibility for 
their own learning). 

In many skill and knowledge areas, few low-income student participants 
were rated as possessing the capacities that teachers deemed desirable for 
academic success, when compared to other students enrolled in the after-school 
program. The same pattern held true for younger versus older students, with 
fewer younger students rated as possessing needed skills. This suggests that after- 
school participants from poor families and young participants have the greatest 
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need for the skill-building and aeademic activities that are offered by NJ After 3 
programs. 



Many parents expressed strong feelings of relief and gratitude that the NJ 
After 3 program was available to their children. In addition to experiencing peace 
of mind, parents reported practical economic and personal benefits, such as the 
ability to focus on their jobs and the freedom and flexibility to attend to necessary 
personal responsibilities. 

To achieve positive life outcomes, students also require opportunities and 
supports in the development of healthful behaviors. Fewer than half of the 
students who responded to the student survey indicated that they engage in 
sustained physical activity for three or more hours each week. The need for 
physically and intellectually stimulating experiences among NJ After 3 
participants was also indicated by student self-reports regarding high levels of 
television viewing and of time playing video games. Student self-reports on 
eating habits also confirmed the need for activities that educate participants on 
nutrition. 



Findings about Important Program Features 

Availability of rich content-based activities. Students and their parents 
want after-school programs to provide information and learning opportunities that 
are not available through their schools and that family resources cannot provide. 
Programs recognize this need and have tried to offer diverse opportunities for 
learning and personal development and to hire appropriate staff for the clubs and 
activities that students and parents have requested. The need is great, however, 
and programs have confronted obstacles such as limited space, unqualified staff, 
or inadequate financial resources as they tried to introduce new classes and 
activities. 

The need to keep students challenged and engaged through the provision 
of rich content-based activities is particularly important in retaining older 
students. In general, fewer students in grades 6-8 experienced the same high level 
of satisfaction with program offerings as did their younger peers, and older 
students attended the after- school programs with less regularity than did younger 
students. Activities that could be developed and expanded to retain older students 
include community service, field trips, conflict resolution programs, health and 
nutrition activities, hands-on science activities, technology, dance, and drama. 

Delivery of learning- and mastery-oriented content. Observation-based 
data indicated that students were particularly engaged by activities that were 
structured and focused on the achievement of clear goals. Long-term projects in 
dance and drama, for example, typically satisfied these criteria and were 
immensely popular among students. Sufficient time should be allotted to 
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minimize disruptions that interrupt the learning experience, and after-school 
program schedules should be developed with this objective in mind. In addition, 
staff should be encouraged to develop programs that allow students to experience 
growth and progress over time and to demonstrate their progress to others. As 
new activities are introduced, those that offer the most potential for learning and 
mastery should be given highest priority. 

Practices to support positive relationships. NJ After 3 programs have 
facilitated the development of positive relationships between staff and students 
and among students. The significant presence of youthful staff members with 
whom students feel they can relate and the programs’ low student-staff ratios 
support these positive relationships. Positive relationships flourish within 
activities that encourage communication between staff and students and that 
interest and engage students physically and mentally. NJ After 3 should continue 
to keep student-staff ratios low and encourage staff to reach out to students to 
initiate new relationships and build trust. Similarly, staff should be intentional in 
their organization and planning of games and activities that require students to 
communicate with and assist each other. 

Strong partnerships. Staff at the NJ After 3 programs have expertise, 
experience, and personal contacts in the fields of both social services and 
education, so they are well positioned to identify resources and information for 
program participants and their families. The result is that many after-school 
programs serve as conduits between families and community resources, helping to 
identify needs, raise awareness, and provide necessary services. 

Much of the interaction between staff and parents occurs informally, such 
as during pick-up at the end of the program day or on the telephone during 
evenings and weekends, when a specific issue arises. However, survey data 
revealed that the amount of communication varies significantly among parents. 
Specifically, regular school-day teachers reported that they communicate with 
African American and white parents more often than they do with Latino and 
Asian parents. As the number of students from Latino and Asian ethnic groups, 
especially recent immigrants, grows in many communities, after-school programs 
need to become more responsive to these populations. While it is not clear from 
our research that language differences explain this finding, such differences may 
be a contributing factor. Most site coordinators reported that a major challenge 
for them was finding sufficient staff with skills to work with English Language 
Learners. 




Students benefit from the efforts of school staff and after-school staff to 
share ideas and materials and to reinforce what each is doing to promote academic 
achievement. However, some programs are still trying to develop procedures and 
establish boundaries and expectations with their host schools. After-school staff 
must continue their efforts to build bridges with school staff. 

Looking toward Year 2. This report describes what we have learned 
about the program goals, participants, staff, activities, operations, and short-term 
outcomes of this initiative based on data collected from Year 1 of the evaluation. 
This year’s descriptions of the programs and students will serve as baseline data 
for the longitudinal analyses that will be conducted in Years 2 and 3 of the 
evaluation. The future phases of the evaluation will be able to provide more 
information on the extent to which programs and participants were able to meet 
the initiative’s goals. 
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Overview of the Evaluation and First-Year Report 

To expand and improve after-school opportunities for children, New 
Jersey After 3 (NJ After 3) has sought to develop and implement a comprehensive 
system of high-quality after-school programs throughout the state. Specifically, 
NJ After 3 seeks to increase the number of after-school programs in the state that 
provide a safe environment for children during after-school hours, offer enriching 
academic activities and homework assistance, and expose children to nurturing 
individuals and meaningful experiences that promote intellectual, physical, social, 
and artistic development. Under an agreement with NJ After 3, Policy Studies 
Associates, Inc., is conducting a three-year, comprehensive evaluation of the 
statewide initiative. 

This report describes what the evaluation has learned about the program 
goals, participants, staff, activities, operations, and short-term outcomes of this 
initiative based on data collected from August 2005 through July 2006. Findings 
are based on an analysis of data obtained from programs initially funded by NJ 
After 3 in 2004-05 (Round I programs) and from programs funded for the first 
time in 2005-06 (Round II programs). 

This first-year report is organized into seven main sections. The first 
section describes the evaluation design and processes in more detail. The second 
section discusses the program-level goals of the initiative and of individual 
programs. The third section describes student participants’ characteristics and 
their program attendance patterns. Section four provides details about programs’ 
activities, structural features, and resources. Sections five and six present baseline 
data on key program and participant outcomes emerging during the first phase of 
this three-year evaluation. Where possible, we offer preliminary answers to the 
study’s research questions and assess the extent to which the initiative is 
progressing toward its goals of: 

■ Expanding the availability of after-school services 

■ Enhancing the quality of after-school services 

■ Creating sustainable programs that are financially secure and of 
high quality 

■ Promoting the health and the emotional, social, and intellectual 
development of New Jersey’s children 

The final section summarizes preliminary findings regarding structures 
and practices that are likely to affect the achievement of program goals. This 
discussion points out program challenges and weaknesses identified by staff, 
students, and parents. 
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1. Evaluation Design and Operations in 2005-06 



This section describes the evaluation design, efforts to obtain informed 
parental consent in the evaluation’s first year, and the data sources used in this 
report. 



Overview of Evaluation Design 

The core task in designing the NJ After 3 evaluation was to devise a 
framework to capture information at baseline and over time that responded to the 
initiative’s goals while accounting for the diversity of NJ After 3 programs and 
participants. The sampling design that we selected allows us to collect certain 
types of overview data from all program sites, with progressively more detailed 
information available from smaller samples. This arrangement helps keep survey 
and data-handling costs in check while permitting the collection of comprehensive 
data on the initiative as a whole. Moreover, the nested sampling strategy permits 
us to make informed estimates of overall program and outcome patterns based on 
information obtained from subsets of programs and participants. Key features of 
the design are as follows: 

■ From all programs in Rounds I and II, the evaluation is collecting 
the following types of data annually for three years. 

Data from NJ After 3’s management information system. 

YouthServices.net, the vendor for the NJ After 3 management 
information system, provides data to us on the characteristics and 
program attendance patterns of all participants. 

Survey data from site coordinators. Site coordinators provide 
data on program goals and activities, program schedules, staff 
recruitment and qualifications, participant outreach and 
recruitment, participant needs and preferences, and efforts to make 
connections among participants’ schools, communities, and 
fa mi lies. 

■ From all programs in Rounds I and II, the evaluation is 
collecting survey data from executive directors in the first and 
third years of the evaluation. Executive directors of provider 
organizations provide data on NJ After 3 program influences on 
the organizations themselves in fulfilling their core missions, NJ 
After 3 programs’ links to other services delivered by provider 
organizations, and the cost and funding of specified elements of NJ 
After 3 programs. 
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From Round I programs, the evaluation is collecting annual 
survey data from student participants for three years. Students 




in grades 3-8 who are also in a Round I program provide 
information on their behavior, attitudes, and skills. 

■ From a sample of 10 Round I programs, the evaluation is 

collecting the following types of data annually for three years. 

Interview and observation data. Site visits to 10 Round I 
programs identified for in-depth study provide interview data from 
site coordinators, program staff, student participants, and parents, 
as well as information on programming. 

Survey data from school-day teachers. School-day teachers in 
the 10 in-depth study sites provide data on the behavior, attitudes, 
and skills of NJ After 3 student participants. ^ We administer 
teacher surveys to school-day teachers of NJ After 3 program 
participants in three grade cohorts. Teachers who can address 
students’ academic and reading/language arts skills are targeted for 
survey response. In Year 1, the survey was administered to 
teachers of participants in grades 3-6. In Years 2 and 3, the survey 
will be administered to teachers of the same sample of participants 
who have remained enrolled in one of the programs in the in-depth 
sample. Because most of these students will have been promoted 
to the next grade, we expect to survey teachers of participants in 
grades 4-7 in Year 2 and then grades 5-8 in Year 3. To help ensure 
an acceptable response rate, each teacher receives $5 from the 
evaluation budget for each survey completed on a single child. 

Using these data sources, the evaluation is addressing these primary 
research questions: 

1 . What are the characteristics of the programs supported by the NJ 
After 3 initiative? 

2. What are the characteristics of the students served by NJ After 3, 
and what are their patterns of attendance? 

3. What did NJ After 3 accomplish with respect to enhancing the 
quality of after-school services in the first three years? 

4. What did NJ After 3 accomplish with respect to expanding the 
availability of after-school services in the first three years? 

5. How successful was NJ After 3 in creating sustainable systems for 
funding and program quality? 



^ The teacher survey adapted and used certain items from the Academic Competence Evaluation 
Scales (ACES), which were developed by James DiPerna and Stephen Elliott for Harcourt Brace. 
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6 . 



What were the initiative’s effects on participating children and 
fa mi lies? 



Efforts to Obtain Informed Parental Consent 

Evaluators worked with NJ After 3 site coordinators and program staff in 
Round I programs to obtain parental consent to collect data from students in these 
programs. Evaluators prepared the consent form in English and Spanish, and NJ 
After 3 posted both versions on the NJ After 3 website. 

■ Erom the Round I programs, we obtained consent forms from 42 
percent of the students in grades 3-8 (1,184 of 2,787 students). 

Of the 2,787 Round I participants in grades 3-8 with data in 
YouthServices.net, the parents or guardians of 909 participants 
gave consent, while parents or guardians of 102 participants denied 
consent. No consent data were recorded for 1,776 Round I 
participants in grades 3-8.^ 

■ Erom the 10 in-depth programs, the evaluation obtained consent 
forms from 62 percent of the students in grades 3-8 (639 of 1,031 
students). 

The consent response rates presented in this report are approximations 
because of challenges faced by evaluators in determining the total number of 
enrolled participants in grades 3-8 at the time of consent administration. In 
particular, programs did not consistently update the YouthServices.net 
management information system to indicate participants’ grade level or school 
assignment. Therefore, it is possible that the reported percent of students for 
whom consent was received is somewhat inflated. This is because we may not 
have counted some eligible students with missing information in the total number 
of enrolled participants (the denominator for the calculation of consent rates). 



Evaluation Data Used in This Report 

This report presents analyses of data collected from programs funded since 
2004-05 (Round I programs) and programs funded for the first time in 2005-06 
(Round II programs). 

Erom all programs in Rounds I and II, we collected the following: 

■ Survey data from 27 executive directors (69 percent of the 
directors of 39 provider organizations) 



^ Because over half of the enrollees received free or reduced price lunch, it is likely that most of 
the parents who did not respond to the request for evaluation consent are low income. 
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Survey data from 55 site coordinators (83 percent of the 
coordinators of 66 programs) 



■ Program attendance and demographic data on 1 1,108 program 
participants in grades K-8 from the YouthServices.net management 
information system 

From Round I programs only, we collected the following: 

■ Survey data from 671 student participants in grades 3-8 (73 percent 
of 919 participants) 

We collected the following from the in-depth sample of 10 Round I 
programs: 



■ Survey data from school-day teachers who assessed 303 NJ After 3 
student participants in grades 3-6 with parental consent (57 
percent of 527 participants) 

■ Interview data from 57 program staff, 63 students, and 23 parents 

■ Observation data obtained from 179 different after-school classes 
or activities during site visits to each of 10 programs 



2. Goals of the NJ After 3 Initiative 

All of the site coordinators who responded to these survey items identified 
the provision of positive adult guidance, a safe environment, and opportunities for 
social development as major program goals, as presented in Exhibit 1. Most (over 
75 percent) also noted that their programs seek to help students improve 
academic, health, and life skills, and provide opportunities for recreation and 
cultural enrichment. About half (29 out of 55, or 53 percent) identified helping 
youth connect to their community as a major goal. 

Executive directors emphasized similar programming goals, with one 
notable exception. Only 48 percent of executive directors (13 out of 27) reported 
that providing recreational activities was a major program goal, while 87 percent 
of site coordinators (48 out of 55) identified this as a major program goal. 

Site coordinators elaborated on their program goals during individual 
interviews and offered additional insight into what they were trying to 
accomplish. Eor example, site coordinators described goals that included 
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Exhibit 1 

Programs Goais Reported by 
Executive Directors and Site Coordinators, 2005-06 




Exhibit reads: Aii responding site coordinators (100 percent) reported that one of the major program 
goais was to provide youth with positive aduit guidance and/or mentors, compared to 96 percent of 
responding executive directors. 



increasing parent involvement and increasing exposure to different types of 
learning and experiences as important goals. Site coordinators’ comments 
included the following: 

“~We provide childcare and enrichment for kids, and we help empower 
parents. We help parents focus on their child’s development, socially, 
physically, and cognitively. ” 

“What the kids learn they bring home and it affects family 
structures .. .When they learn chess or dance and [bring it home] and teach 
their brother [while mother watches [. . . it has a real effect on the whole 
family. ” 

“Providing students with different forms of learning. Children are burned 
out academically, and many times the recreation components are down [in 
school]. Children can learn in different ways [through] music, dance, and 
different things in non-traditional ways. ” 
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“They can do it and you need to push them and see what they can do 
because they need the challenge. Instead of 1 plus 1, they are doing 12 
plus 12. ” 

Other frequently articulated program goals included reaching students who 
are in unsafe or at-risk environments, increasing homework completion rates, and 
improving academic and social skills. 

“I could have a little Malcolm X in front of me. . .1 get tired of [seeing 
wasted] potential everywhere. 1 drive by the drug dealers on my way 
home. 1 want to prepare students [to succeed in life]. ” 

“1 am told that 30 percent of the kids in this building do not do their 
homework, so that is what we focus on. ~We get students who are in the 
bottom third of the school... the child that is in need of the service. ” 



The diversity of goals and variation in emphases among site directors and 
executive directors reflect the fact that after-school programs serve student 
populations with multiple competing needs. While there has apparently been an 
increased emphasis on academic goals in recent years (e.g., 59 percent of 
executive directors who have been operating after-school programs for several 
years said their focus is “somewhat” or “much more” on academic programming 
than before), many site coordinators and staff members were concerned about this 
trend, fearing it places undue pressure on children. In response, many said they 
were becoming much more intentional in their efforts to balance traditional 
academic activities with other types of intellectually stimulating programming 
and physical play. 



3. Participants’ Demographic Characteristics and 
Program Attendance 

The majority (51 out of 55, or 93 percent) of site coordinators reported 
that they sought to serve all interested students, as shown in Exhibit 2. However, 
many also reported that they targeted students who had been recommended by 
school teachers and counselors (62 percent) or youth who had been recommended 
and identified by their school as being in need of special assistance in reading or 
mathematics (56 percent). 

A total of 1 1,108 students in grades K-8 participated in NJ After 3- 
supported programs during the 2005-06 academic year. While each program 
served an average of 150 students, that number varied widely, from a minimum of 
about 50 students to a maximum of over 300 students in a single program. 
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Exhibit 2 

Types of Participants Targeted by NJ After 3 Programs, 2005-06 




Exhibit reads: Ninety-three percent ef responding site coordinators reported that the program 
sought to serve aii interested youth. 



Almost identical numbers of boys (5,568 of 1 1,101, or 50 percent) and girls 
(5,533 of 11,101, or 50 percent) were served through the initiative, as shown in 
Exhibit 3. Eighty-three percent (8,107 of 9,734) were either African American 
(4,772 of 9,734, or 49 percent) or Eatino (3,335 of 9,734, or 34 percent). 
Approximately half (5,791 of 10, 975, or 53 percent) were eligible to receive free 
or reduced-price lunch. Ear fewer student participants were English Eanguage 
Eearners (1,333 of 10,963, or 12 percent) or received special education services 
(231 of 10,703, or 2 percent) during the school day. In general. New Jersey After 
3 participants were more likely to be enrolled in grades K-2 or grades 3-5 than in 
grades 6-8, as shown in Exhibit 4. 

As part of our analysis, students were categorized into one of three levels 
of participation, based on the number of days they attended the program and 
received services. Students in grades K-8 were identified as “highly active” if 
they attended a NJ After 3 program for at least 80 days and attended at least 80 
percent of the days that they were enrolled in the program during the school year. 
An “active” participant was identified as one who attended at least 60 days and 
attended at least 60 percent of the days during which they were enrolled. “Non- 
active” participants were those who attended fewer than 60 days or less than 60 
percent of the days during which they were enrolled. Attendance rates were 
calculated based on the number of days each student attended their NJ After 3 
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program since their date of enrollment and the number of days it was possible for 
the student to attend. 



Exhibit 3 

Comparison of Demographic Characteristics by Participation Levei, 2005-06 







Percent of Students in Grades K-8 


Measure 


Highiy 

Active 

Participants 

(n=4,755) 


Active 

Participants 

(n=2,252) 


Non-Active 

Participants 

(n=4,101) 


Aii 

Participants 

(n=11,108) 


Gender (n=11,101) 




Male 


49 


49 


52 






Female 


51 


51 


48 




Race/ethnicity (n=9,734) 




Hispanic 


29 


35 


40 


34 




African American 


52 


51 


44 


49 




Asian or Pacific 
Islander 


1 


1 


1 


1 




White 


17 


13 


14 


15 




American Indian or 
Native American 


1 


0 


0 


1 


Free or Reduced Priced Lunch (n=1 0,975) 




Yes 


54 


52 


52 


53 




No 


20 


16 


15 


17 




Unspecified 


26 


32 


34 


30 


Limited Engiish Proficiency (n=1 0,963) 




Yes 


11 


12 


14 


12 




No 


67 


58 


53 


60 




Unspecified 


22 


30 


32 


28 


Speciai Education Status (n: 


=10,703) 










Yes 


2 


2 


3 


2 




No 


49 


37 


38 


42 




Unspecified 


50 


61 


60 


56 



Exhibit reads: Overail, 50 percent of participants were maie. Among maie participants, 49 percent 
were highiy active, 49 percent were active, and 52 percent were non-active. The 1 1 ,108 students 
on whom we have demographic characteristics comprise the fuii NJ After 3 popuiation. 
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Exhibit 4 

Distribution of Enroiied Students, by Grade, 2005-06 



Grade in 
2005-06 


Number of enrolled 
students 


Percent of 
NJ After 3 Total 


K-2 


3,958 


36 


3-5 


4,124 


37 


6-8 


3,026 


27 


Total 


11,108 


100 



Exhibit reads: On average, 36 percent (3,958) of the students in the NJ After 3 
initiative were in grades K-2. The 1 1 ,108 students on whom we have 
enrollment data comprise the full NJ After 3 population. 



Students who attended NJ After 3 programs did so with a high degree of 
frequency. On average, students across all grade levels attended the program 
almost three-quarters (73 percent) of the days that it was possible for them to 
attend, as shown in Exhibit 5. These levels are consistent with the attendance 
rates of similar high-quality after-school programs, such as those of the After- 
School Corporation (TASC) in New York City. Attendance in NJ After 3 
programs was higher among younger students (e.g., for students in first grade, 
attendance was 80 percent of possible days) than among older students (e.g., for 
students in eighth grade, attendance was 58 percent of possible days). 



Exhibit 5 

Distribution of Days Attended, by Grade, 2005-06 



Grade in 
2005-06 


Average Number of Days 
Students Attended NJA3 
Program 


Average Attendance 
Rate 


K 


106 


77% 


1 


108 


80% 


2 


105 


78% 


3 


101 


77% 


4 


102 


76% 


5 


96 


73% 


6 


83 


69% 


7 


63 


59% 


8 


62 


58% 


Overall Average 


94 


73% 



Exhibit reads: On average, participants in kindergarten attended the program 106 
days. Their average attendance rate was 77 percent. The attendance data are 
based on the full NJ After 3 population (n=1 1 ,108). 
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While there were significant variations in the attendance rates of students 
in different grades, the average attendance rates were similar for students who 
attended Round I and Round II programs. 

From all programs in Rounds I and II, participants in a majority of 
programs (49 of 58, or 84 percent of programs) attended their program at a rate of 
60 percent or higher. In fact, participants in 22 programs attended their program 
at an average rate of 80 percent or higher, and participants in 27 programs 
attended at an average rate of 60 to 79 percent. Six of the remaining programs 
had average attendance of 50 to 59 percent, and three programs had average 
attendance of 49 percent or less. These program-level figures represent the 58 
programs that reported their participants’ program attendance in 
YouthServices.net and exclude programs with incomplete data. 

In 2005-06, 43 percent of K-8 participants met the evaluation’s criteria for 
highly active participation, and 20 percent met the criteria for active participation, 
as shown in Exhibit 6. 



Exhibit 6 

Distribution of Participants, by Participation Levei and Grade, 2005-06 



Grade in 
2005-06 


Percent of Students in Grades K-8 


Highly Active 
Participants 

(n=4,755) 


Active Participants 

(n=2,252) 


Non-Active 

Participants 

(n=4,101) 


K 


50 


22 


28 


1 


53 


24 


24 


2 


51 


21 


28 


3 


48 


19 


33 


4 


47 


22 


31 


5 


45 


18 


37 


6 


34 


21 


45 


7 


22 


16 


63 


8 


20 


19 


61 


Overall Average 


43 


20 


37 



Exhibit reads: Among participants in kindergarten, 50 percent were highiy active, 22 
percent were active, and 28 percent were non-active. The participation data are 
based on the fuii NJ After 3 popuiation (n=1 1 ,108). 



In the majority of programs (40 of 58, or 69 percent), at least 50 percent of 
participants met the evaluation’s criteria for active or highly active participation. 

In 26 of these programs, 75 percent or more of participants met criteria for highly 
active or active participation. In 15 programs, 25 to 49 percent of participants met 
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active or highly active participation criteria. In three programs, 24 percent or less 
met these criteria. 



4. Overview of NJ After 3 Programming 

This section describes NJ After 3 activities and programming through 
discussions focused on program content, program context, staff characteristics, 
staff training and benefits, relationships between staff and students, relationships 
with parents and the community, and relationships with the schools. 



Program Content 

In 2005-06, NJ After 3 programs offered a wide array of academic, 
artistic, social, health, civic, and athletic activities that attempted to address needs 
identified by staff and to appeal to students and their parents. Many site 
coordinators said that they sought to do two things: build student skills and 
maximize students’ exposure to new and different activities. Over half of those 
we interviewed acknowledged, however, that exposure was the more important 
goal. One site director reflected his peers’ commonly held sentiment that children 
need exposure to the possibilities outside their immediate environment: “The 
world does not stop at [our town]. A lot of [our] students have not even been over 
the bridge to [the neighboring city] .” In those instances when skill development 
was emphasized, it tended to employ content related to the arts (e.g., jewelry 
making) and technology (e.g., creating a web site). 

The emphasis on exposure over skill building or mastery also reflected the 
staff’s recognition of inherent limitations. “Why would you think that in three 
hours a day a kid is going to master a skill? Maybe it’s possible but that is not my 
expectation. My expectation is to provide after-school programming where kids 
can learn and simultaneously have fun. I want safe exposure for these children.” 

Parent interviews indicated that parents were particularly pleased that NJ 
After 3 provided a wide and interesting range of activities for children. “I like the 
concept of it,” said one parent. “Even though they have homework time, they 
have time for the kids to be themselves.” Another added that her son was now 
able to participate in “activities that he might otherwise not have tried. . .he is 
much more active.” All of the students whom we interviewed also expressed 
satisfaction with the program overall. Many indicated that the after-school 
program allowed them to learn a skill or participate in an activity that they had 
never before tried. Favorite activities included art, computers, gym, martial arts, 
and drama. Special clubs such as chess and music or clubs that encouraged 
students to discuss their feelings and express their thoughts about a range of 
personal issues were also very popular. 
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As a group, site coordinators reported a fairly consistent set of academic 
program offerings across the sites. All or nearly all of the programs offered 
opportunities for students to do their homework, participate in math and other 
learning games, and practice reading and writing, as shown in Exhibit 7. 



Exhibit 7 

Types of Academic Activities Offered 
by NJ After 3 Programs, 2005-06 




Exhibit reads: Aii responding site coordinators (100 percent) reported that their after-schooi 
program offered iearning games and activities for youth. 



During our interviews we learned that interest in computers and 
technology was high among many students. However, in some sites only the 
older students had access to technology activities. In other locations, computers 
were either not available to anyone or, as one student told us, “they are all 
broken.” In sites where computers were accessible, they were used in various 
ways. For example, students at one site reported that they were taking a video 
production elective. “We do editing,” explained one participant. “A couple days 
ago we interviewed random people coming out of classes [on camera].” Others 
described classes in which they played computer games or practiced typing. 
“When we have computers, sometimes we do mini quizzes about singers and 
rappers and we play tanks. . .and [computer games about] robots. Sometimes we 
. . .learn to type without looking.” 

Students also expressed a great deal of enthusiasm for arts-related 
activities, as shown in Exhibit 8, particularly arts activities that required 
rehearsing and preparing for major events that allowed them to showcase their 
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talents in front of their peers, parents, and other members of the community. 
These included fashion shows, plays, dance routines, and long-term visual art 
projects (e.g., kite and mask making). “We’re trying to open their eyes so they 
can see there a lot of things out there that they can learn. There’s art, yes, but 
there’s also sculpture and painting and music... [We are trying to expose them] to 
a lot of things,” explained one site coordinator. 



Exhibit 8 

Artistic and Sociai Deveiopment Activities Offered 
by NJ After 3 Programs, 2005-06 




Exhibit reads: All responding site coordinators (100 percent) reported that their after-school 
program offered visual arts and crafts activities for youth. 



Programs offered certain activities in the areas of civic engagement, 
community service, and career exploration, as shown in Exhibit 9. Eighty-five 
percent of site coordinators (47 of 55) reported that they administered service 
projects in the program, such as participants tutoring other students, clean-up 
projects, or participants helping program staff lead an activity. Eewer site 
coordinators (25 of 55, or 45 percent) reported that their program offered service 
learning projects that involved the neighborhood, such as visiting the elderly or 
community clean-up projects. 

In some programs, students joined the 4-H Club in order to develop public 
speaking skills and civic awareness, while in other programs, students participated 
in shorter-term projects such as food- or clothing-drives for the homeless or sang 
at local senior citizen residences. 
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Exhibit 9 

Civic Engagement, Community Service, and Career Expioration 
Activities Offered by NJ After 3 Programs, 2005-06 



Civic, community, and career activities offered 

Service projects in the program 

Discussions of current nationai or worid events 
Activities where students discuss current events in their 
communities 

Career expioration activities 
Service projects in the neighborhood 
Civic simuiation games 
internships 
Job shawdow ing 

0 20 40 60 80 100 

Percent of site coordinator survey respondents 

■ Site coordinators (n=55) 

Exhibit reads: Eight-five percent of responding site coordinators reported that their after-schooi 
program offered service projects within the program (e.g., tutoring other students, ciean-up 
projects, heiping program staff iead an activity). 




During interviews, students spoke quite positively about those programs 
that focused on athletics, health, and life skills, as shown in Exhibit 10. Older 
students in particular appeared to enjoy those activities that allowed them to 
participate in peer discussions about social and personal issues. These classes 
were referred to by different names (e.g.. Smart Girls, Free Teen USA, I Can 
Problem Solve) at each location, but they generally aimed to educate students 
about the danger of drugs, smoking, and premarital sex. They also aimed to instill 
values such as integrity and discipline, and taught strategies that will enable 
participants to solve routine social problems and issues that pre-teens often 
experience. Role-play, games, writing exercises, and debate were used to engage 
students in these activities. 

Nutrition education and opportunities for exercise were also available at 
most sites, as shown in Exhibit 10. Programs offered free play, organized sports 
and games, gymnastics, martial arts, and structured physical fitness activities 
through curriculum-based programs such as Spark and KidFit. 

Staff and student participants noted in interviews students’ need for 
increased recreation and physical activity, and they often alluded to the absence or 
minimal amount of time available for recess and play in many schools. “They 
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really enjoy gym,” said one program staff member. “They get to run around like 
crazy because they don’t get to run around otherwise.” Another staff member 
lamented students’ lack of recreational opportunities during the school day. 
“They only have gym one day [per week] for 90 minutes, and they have to get on 
a bus to go to the gym. There’s no playground [at the school], so there’s not too 
much outlet for them.” 



Exhibit 10 

Athietic, Heaith, and Life Skiiis Activities Offered 
by NJ After 3 Programs, 2005-06 




Exhibit reads: Ninety-six percent of responding site coordinators reported that their after-schooi 
program offered free time for recreation for youth. 



Fewer than half of the students who responded to the student survey 
indicated that they engage in sustained physical activity lasting three or more 
hours each week, as shown in Exhibit 1 1 . Only 42 percent of participants (264 of 
628) in grades 3-8 reported that they played basketball, football, or soccer at least 
three hours per week. Even fewer (245 of 609, or 40 percent) reported that they 
ran or walked or that they danced as part of a group (155 of 605, or 26 percent) 
for three or more hours each week. 

Students in grades 3-5 were more likely to report that they participated in 
sustained physical activity than were older students in grades 6-8. Eor example, 
while 21 percent of students in grades 3-5 (88 of 429) reported that they engaged 
in martial arts like Taekwondo or judo at least three times per week, only 6 
percent of youth in grades 6-8 (10 of 176) reported that they did the same. 
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Similarly, 29 percent of students in grades 3-5 (124 of 426) reported that they 
danced as part of a group at least three times per week, compared to 17 percent of 
students in grades 6-8 (31 of 179). 



Exhibit 11 

Percent of Students Engaged in Physicai Activity 
Three or More Hours Per Week, by Grade Ciuster, Grades 3-8, 

2005-06 



Participating 3 or More Hours 
per Week in: 


Percent of Students 


Grades 3-5 

(n=445) 


Grade 6-8 

(n=183) 


Aii Grades 

(n=628) 


Basketball, football, or soccer 
(n=628) 


45 


37 


42 


Running or walking (n=609) 


43 


35 


40 


Martiai arts (n=605)* 


21 


6 


16 


Dance as part of a group (n=605)* 


29 


17 


26 



* The differences were statistically significant (p<.05). 



Exhibit reads: Acress grades 3-8, 42 percent ef participants participated three or more hours per 
week In basketball, football, or soccer. Ferty-five percent of participants In grades 3-5 reported as 
such, compared to 37 percent of students In grades 6-8. 



We found no significant relationship between the level of physical activity 
reported by students and their level of participation in the after-school program. 

The need for physically and intellectually stimulating experiences among 
NJ After 3 participants was also indicated by student self-reports regarding their 
television and video game habits. More than one-third of the participants surveyed 
watched four or more hours of television per school day, as shown in Exhibit 12. 
One out of five students reported that they played video games for a similar 
amount of time. Students’ level of participation in after-school programs was not 
significantly related to how much time they spent in these inactive behaviors. 



Exhibit 12 

Students’ Engagement in Inactive Behaviors 
Four or More Hours Per Schooi Day, by Grade Ciuster, 2005-06 



Participating 4 or More Hours 
per Schooi-day in: 


Percent of Students 


Grades 3-5 

(n=435) 


Grade 6-8 

(n=181) 


Aii Grades 

(n=616) 


Watching television (n=610) 


39 


42 


40 


Playing video games (n=616) 


23 


17 


21 



Exhibit reads: Across grades 3-8, 40 percent of participants watched television four or more hours 
per school day. Thirty-nine percent of participants in grades 3-5 reported as such, compared to 
42 percent of students in grades 6-8. 
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Program Context 



In addition to examining the range of program offerings, the evaluation 
also assessed the context within which these activities take place by observing 
179 different activities or classes during site visits to each of 10 programs. 
Specifically, we examined student engagement, instruction, activity content, and 
staff and student relationships in program activities, using a detailed observation 
guide. 



To assess student engagement in program activities, we examined the 
extent to which students remain on-task, listen to peers and staff, contribute to 
discussions, make meaningful choices, take leadership, or assume responsibility 
during the activity. We observed significant variation in student engagement 
measures across certain activities. For example, students scored low on these 
student engagement measures during homework activities. This may be because 
homework activities typically required students to work alone and in silence on a 
task assigned by a teacher. In most instances, communication with others during 
this activity was actively discouraged. Similarly, students engaged in sports 
activities had few opportunities to “contribute their ideas to discussions” mainly 
because few discussions were held during sports activities. Students scored 
higher on student engagement measures in other academic and enrichment 
activities. In arts activities in particular, students tended to communicate and 
listen to each other more often, and they were typically allowed to make more 
meaningful choices within the context of the activity (e.g., assisting with 
choreography during a dance class). 

We developed an activity scale, based on assessments of key aspects of an 
activity such as content and structure of the activity, including organization, 
challenge level, analytic thinking, and skill involvement. Those activities that 
targeted decision-making or interpersonal communication skills (e.g., many of the 
character-building programs and peer discussion groups) received higher scores 
on the activity scale compared to other kinds of activities. Highly rated activities 
were typically well organized, challenging for students, and required analytic 
thinking. Academic activities (other than homework) received the highest score 
on the activity scale and also demonstrated strong staff-student relationships and 
varied instructional strategies. 

Student-staff relationships were found to be stronger during those 
activities in which staff were focused on building students’ skills — regardless of 
the content of the activity — when compared to activities in which staff were 
encouraging students to practice a skill or to do homework. Similarly, student- 
staff relationships were also stronger in activities that targeted decision-making or 
interpersonal communication, as compared to other activities. These results are 
not surprising given that the latter activities tend to be driven by discussion and to 
have a youth-development focus. Likewise, skill-building activities typically 
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require more sustained student-staff interactions than homework or other 
activities in which students work independently. 

We observed few instances of collaboration among youth, which we 
defined as youth working together or sharing materials to accomplish tasks in 
which they were equal partners. When it did occur, it was usually during sports 
activities where students were sometimes required to strategize and work together 
in order to win. While there was some evidence of students assisting each other 
(e.g., one student explaining an assignment to another or helping a friend to spell 
a word), particularly during homework, across all activities, these incidents were 
rare. 



We found that the presence of a certified teacher leading an activity had 
no significant relationship with the type of instructional methods that were 
employed during the activity or on any other components of the activity scale. 
Research suggests that in order to engage students with different learning styles 
staff should diversify instructional methods. By instructional methods, we refer 
to the teaching approaches that are often employed by classroom teachers, such as 
direct instruction, coaching, modeling, and demonstration. Direct instruction 
includes lecture and didactic questioning; in coaching, the teacher supports and 
facilitates the performance of a student in carrying out a task to achieve mastery; 
modeling consists of exhibiting the behavior or activity that the teacher wants the 
student to perform; and demonstration is similar to modeling but involves more 
explanation and discussion of what is going on. Nevertheless, the data suggest 
that, when the goal of an activity was to build or practice skills, staff were more 
likely to employ clear-cut and varied instructional methods than were used during 
homework or other activities. 

These observations suggest that students’ experiences during the after- 
school program varied widely depending on the activities in which they were 
engaged. For example, while opportunities for thinking analytically, developing 
relationships with staff, contributing to discussions, making meaningful decisions, 
assuming leadership roles, and collaborating with peers may exist at a single 
program, it is unlikely that a staff member would be able to provide all these 
experiences during a single activity. This suggests that exposure to a variety of 
activities, each with a different focus (e.g., skill-building, skill-practicing, 
interpersonal communication), maximized students’ opportunities to learn and 
promoted well-rounded development. 



Staff Characteristics 

The after-school staff employed by NJ After 3 programs varied in many 
respects. This section examines the knowledge and experiences they brought to 
their work, how they were assigned within the program, and other aspects of their 
employment. 
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The average number of staff employed per program was 20, ranging from 
a minimum of three at one site to 45 at another. Almost half (44 percent) had 
bachelors degrees or higher, 19 percent were enrolled in college, 25 percent had a 
high school diploma or less, and 13 percent were teenagers, as shown in Exhibit 
13. In addition, approximately one-quarter (26 percent) of all adults employed by 
the programs were certified teachers. 



Exhibit 13 

Educationai Quaiifications of Program Staff, 
According to Site Coordinators, 2005-06 



Educational Qualifications 


Average Percent of Staff 

(n=55 ) 


Had B.A. or B.S. degrees or higher and had teaching 
certificates 


26 


Had B.A. or B.S. degrees or higher but not certified to 
teach 


18 


Were current college students 


19 


Were teen staff or high school students 


13 


Had a high school degree or less and were not currently 
enrolled In college 


25 



Exhibit reads: According to site coordinators, 26 percent of program staff had bachelors degrees or 
higher and had teaching certificates. 



Only 10 percent of the staff worked for the after-school program on a full- 
time basis, with an additional 15 percent employed half-time for 20 to 34 hours 
per week, as shown in Exhibit 14. The largest number of employees (75 percent) 
worked part-time, or less than 19 hours per week. Although the majority of staff 
members (87 percent) worked directly with students, the proportion of full-time 
employees who were engaged in direct service was significantly smaller (36 
percent). Among half-time employees, the percentage engaged in providing 
direct services was 50 percent. A large majority (88 percent) of part-time 
employees also worked directly with students. 
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Exhibit 14 

Empioyment Hours by Staffing Category, 2005-06 



Staff Categories 


Percent of Staff by Hours of Employment 

(n=49) 


Average 
Percent of 
All Staff 

(n=52) 


Full-time 

(35 hours or 
more per week) 


Half-time 

(20-34 hours 
per week) 


Part-time or 
less 

(less than 19 
hours 
per week ) 


Administrative staff, such as 
assistant director, parent coordinator 


50 


32 


6 


8 


Support staff, such as administrative 
assistants 


14 


18 


6 


6 


Direct service staff who worked 
directiy with participants 


36 


50 


88 


87 


Overall Average 


10 


15 


75 


100 



Exhibit reads: On average, 8 percent of the program staff consisted of administrative staff. Fifty 
percent of the staff who worked fuii-time, 32 percent of the haif-time staff, and 6 percent of the part- 
time staff were administrative duties. 



The student- staff ratios observed during the site visits, as described in 
Exhibit 15, indicated that the programs were operating well within the parameters 
established by NJ After 3 (student-staff ratios of no more than 10:1 for grades K-5 
and 11:1 for grades 6-8). This finding was further supported by the site 
coordinator survey data that showed that most site coordinators (50 of 55, or 91 
percent) believed that student groups were small enough to allow staff to meet 
participants’ individual needs. 

NJ After 3 site coordinators were typically well-educated and experienced, 
with nearly all (51 of 55, or 95 percent) having worked at least one year or more 
as an after-school program director or staff member, as shown in Exhibit 16. A 
similar number (50 of 55, or 91 percent) had at least a four-year college degree, 
and more than a quarter (15 of 55, or 27 percent) were certified teachers. 
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Exhibit 15 

Staff Ratios in Observed Activities, 2005-06 



Observed Activity 


Student to Staff Ratio 


By content area 


Arts-visual and performing 


5:1 


Academic enrichment 


6:1 


Homework help or tutoring 


8:1 


Sports and games 


8:1 


By skiii targeted 


Skill-building 


6:1 


Skill-practice or reinforcement 


6:1 


Homework or tutoring 


8:1 


By grade ievei 


Grades K-2 


5:1 


Grades 3-5 


7:1 


Grades 6-8 


6:1 


Overaii Average 


6:1 



Exhibit reads: The student to staff ratie in ebserved arts activities was, on average, 5:1 , based on 
observation data obtained frem 1 79 different ciasses or activities during site visits. 



Exhibit 16 

Quaiifications of Site Coordinators, 2005-06 



Quaiifications 


Percent of Site Coordinators 

(n=55) 


More than one year of work experience as a ... 


Staff member in an after-school program 


57 


Program director at an after-school program 


38 


Recreation, youth, or childcare worker 


35 


School administrator 


6 


Highest ievei of education 


Some college 


6 


Completed two-year college degree 


4 


Completed four-year college degree 


56 


Some graduate work 


22 


Master’s degree or higher 


13 


Certification 


Teaching certificate 


27 



Exhibit reads: Fifty-seven percent of responding site coordinators reported that they had more 
than one year of work experience as a staff member in an after-schooi pregram. 
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staff Training and Benefits 

Nearly all (26 of 27, or 96 percent) of the executive directors surveyed 
reported that their provider organizations provided some type of training to site 
coordinators or other staff members. Ninety-three percent (25 of 27) specified 
that they provided training in youth development, and the same percentage also 
said they provided training in curriculum development. A similarly high 
proportion (24 of 27, or 89 percent) reported that they provided training in 
program and staff management. 

Based on our interviews with program staff, it appeared that programs 
differed in the amount of training that they required staff to engage in and that 
they made available to staff. For example, staff at one program indicated that 
some staff were required to receive 20 hours of training per year, while those with 
a different job title — but who also worked directly with students — were required 
to receive 12 hours of training. Staff at another site told us they had received no 
more than about 8 hours of training during the year, and those at a third program 
indicated that they did not know of any training requirement. 

These variations help to explain an apparent contradiction in our survey 
data: Although almost all of the executive directors reported that training was 
provided, a sizable minority of site coordinators (23 of 55, or 41 percent) 
identified limited professional development for staff as a major challenge to 
program quality. Together, these data seem to suggest that access to professional 
development and training was uneven among staff and programs. This may be 
due, in part, to the fact that state licensing changed the requirements for staff 
training in fall 2006, creating some confusion. The number of hours of training 
required differs by staff position. For example, site coordinators are required to 
attend 20 hours of training, while youth development workers are required to 
participate in nine hours. 

When asked to select the training topics of most value to their after- school 
program staff, 89 percent of site coordinators identified classroom management, 

84 percent said youth development training, and 82 percent selected training in 
academic enrichment and literacy, as shown in Exhibit 17. For themselves, 
approximately two-thirds (67 percent) of the site coordinators said training in staff 
supervision would be useful. A similar number (66 percent) selected training in 
youth development, while 62 percent identified program management and 
program design as useful. 
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Exhibit 17 

Training Topics Reported to be Usefui to Staff, 2005-06 



Training topics that are usefui to staff 

Classroom management 
Youth development 
Academic enrichment and literacy development 
Health promotion/education and support services 

Athletic Instruction 
Fine and performing arts 
YouthServices.net 
Staff supervision* 

Program management* 

Program design* 

0 20 40 60 80 too 

Percent of site coordinators reporting that the training topic is usefui 

n Useful to site coordinators (n=55) □ Useful to other staff (n=55) 

*These were training topics that were useful to site coordinators only. 

Exhibit reads: Eighty-nine percent of site coordinators identified classroom management as 
among the most useful training topics for their staff. For themselves, 46 percent of site 
coordinators said that classroom management was a useful training topic. 
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In addition to formal training events, staff reported that they also received 
professional development and training informally during staff meetings and in 
individual meetings with their site coordinator. Others noted that their peers 
served as mentors and guides and that assistance was sought and provided on an 
as-needed basis. 

In terms of employment benefits, a majority of full-time staff who worked 
in the after-school program received leave benefits and were paid to attend 
training events such as conferences and other meetings. In addition, most had a 
retirement savings plan through their employer. Most part-time staff were also 
paid to attend professional development events, but very few received other 
employment benefits. 
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Relationships between Staff and Students 

An overwhelming majority of the parents who were interviewed expressed 
satisfaction with program staff, often describing them as trustworthy, committed, 
capable, caring, and fair. For example, one mother described an incident in which 
her son was punished by a staff member. The staff member instructed him to 
write a brief essay on why he should not have done what he did, and she also told 
him to identify other more appropriate actions he could have taken. “They made 
him think about what he had done. It became a public speaking lesson, a 
responsibility lesson. . .1 just really liked how they handled it,” the parent said. 

Many parents also emphasized that they liked the fact that the staff 
members were relatively young, because they believed this enabled staff to 
connect with children more easily. “They look at [the staff] as friends because 
they are closer in age [and] staff play more of a big sister or big brother role. 

[The students] look up to them.” Several parents also attributed their child’s 
social development and improvements in confidence and self-esteem to the 
family-like atmosphere they said was fostered by program staff. 

Among the many positive sentiments expressed by parents regarding the 
relationship between staff and students were the following: 

“The city needs more programs like these. There is a trust level with the 
people here. . .My daughter is happy and that makes me happy. I believe 
staff [are] pretty clear [about] their goals [and] they are committed to 
achieving them. ” 

“]My son] was a first-grader. He told his teacher ‘My mommy said I 
don’t have to go to aftercare. ’ [He] walked home [and the site 
coordinator] wound up at my doorstep to get my son. That showed me 
that it’s not just a job for her. She did not have to do that. ..she could just 
have made some phone calls. She was so upset, like it was her child. She 
loves her job and these kids. ’’ 

“I like the balance that they have [on staff] between males and females. 
Especially for young men who do not have fathers around, like my son. 
That is very important. ’’ 

“Some kids don’t have [a young adult] to talk to [who] is not their parent. 
[The staff] do not betray [the children’s] trust, but they are like a 
gateway. . . ] they help parents ] know how the child is doing. I can ’t say 
enough about them, I just can ’t. ’’ 
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“I think that by having [staff [who are just a little bit older. ..they can go 
to [them] if they don’t feel comfortable saying [something [ to their own 
mom. They feel more comfortable talking to counselors here [at the after- 
school program [ than the school counselor. ” 

The students with whom we spoke during our site visits echoed these 
positive sentiments. “The after-school program will welcome you with open arms. 
They’re like family. You can talk to them,” said one student. Another added: 
“My family is crazy, each person has a different personality. The staff [here] are 
like that. [The site coordinator] is like the mom [in the family.]” Yet another 
summed up his view of the staff this way: “They’re not only here for academics; 
they’re here to get to know us.” 

Student survey data also indicated that a majority of students viewed 
program staff in positive ways. Over three-quarters of the after-school 
participants reported positive interactions with NJ After 3 staff members, as 
shown in Exhibit 18. Across all grade levels, 85 percent of participants agreed 
that staff really cared about them, and 78 percent agreed that staff cared what they 
think. Seventy-six percent indicated a high level of trust by reporting they can 
talk to staff about things that were bothering them. 



Exhibit 18 

Student Perceptions About Staff, Grades 3-8, 2005-06 



In this after-school program... 



Staff think I can do things w eii 

Staff treat me w ith respect 

Staff try to be fair 

Staff reaiiy care about me 

Staff heip me to try new things 

Staff care w hat i think 

Staff often keep their promises 
i f eei that I can taik to staff about things that are bothering 
me 

0 20 40 60 80 100 

Percent of participants who reported they "agree" or "agree a lot" 



n Ail responding participants (n=633) 

Exhibit reads: Across grades 3-8, 88 percent of responding participants agreed or agreed a lot that, 
in the after-school program, staff think they can do things well. 
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Feedback from parents and students during the site visits also indicated 
that most students had close friends in the program and that, in general, 
relationships among participants were good. Student survey data supported these 
findings. Over three-quarters of the after-school participants reported positive 
perceptions about their peers on almost all of the survey questions pertaining to 
peer relationships, as shown in Exhibit 19. Fewer students (60 percent) said that 
they could really trust their peers. 



Exhibit 19 

Student Perceptions About Peer Reiationships, Grades 3-8, 2005-06 

In this after-school program I... 



Have a good time playing w ith other kids 
Get to know other kids really w ell 
Have a lot of friends 
Get along w ith other kids 
Get to w ork w ith other kids as part of a team 
Like the other kids 
Can really trust the other kids 




0 20 40 60 80 too 

Percent of participants who reported they "agree" or "agree a lot" 



I All responding participants (n=630) 



Exhibit reads: Across grades 3-8, 89 percent of responding participants agreed or agreed a iot that 
in the after-schooi program they had a good time piaying with other kids. 



Relationships with Parents and the Community 

Many of the parents who were interviewed for the study indicated that 
they had not been very involved in the program. Most cited work schedules as the 
primary reason for their lack of involvement. Those who had been able to 
volunteer their time had reportedly chaperoned trips and other events, or they had 
prepared and served food at various celebrations. A few parents also reported that 
they had intentionally arrived early for their child on occasion in order to observe 
activities and to gain a better understanding of the program. “I have actually 
been present during the program,” said one parent. “I came early and I like to do 
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that every now and then. I sat in on [one activity] . . .1 wanted to see how [staff and 
students] interacted and I was very impressed.” 

Programs reached out to parents and the community in various ways. 
Although all programs held events and reached out to parents and the community, 
site coordinators reported that they made some connections more frequently than 
others. At least once every month, almost all of the site coordinators said that 
they reached out to parents by calling home, arranging opportunities for 
communication with representatives from local agencies, and meeting with one or 
more parents, as shown in Exhibit 20. Nevertheless, a few parents indicated that 
they believe the program can do a better job of communicating with them. One 
parent, for example, suggested that the program mail newsletters to the home 
rather than sending correspondence via the students. Another parent added that 
staff should communicate with them when there is a need. “Don’t assume that 
I’m so busy that you can’t ask me [to participate]. Let me know that you need 
things. . .if you need something, don’t be afraid to ask me.” 



Exhibit 20 

Program Reiationships with Parents and the Community, 2005-06 



Connections that are made at least once a month... 



Site coordinator has conversations writh parents over the phone 

Families have the opportunity to hearfrom and talk with 
representatives from local agencies or other organizations (e.g., health, 
police, employment and training programs) 

Site coordinator meets with one or more parents 
Materials about the program are sent ho me to parents 

Events o r meetings are held to which parents are invited 

Events at the program are held (e.g. meetings, performances, 
tournaments, open house, etc.) 

Events or meetings are held to which community members are invited 




0 20 40 60 80 

Percent of site coordinator survey respondents 



□ Site coordinators (n=55) 



■100 
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100 



Exhibit reads: Aii responding site coordinators (100 percent) reported that they had conversations 
with parents over the phone at ieast once every month. 



While the after- school programs did not rely heavily on parental 
involvement, at least half of all site coordinators identified three ways in which 
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local community agencies and other external organizations provided valuable 
resources that supported the goals of the after-sehool program, as shown in 
Exhibit 21. The resources were: special programs, activities, and serviees that 
were provided for youth on- or off-site; funding through grants or eontracts; and 
referrals of students to the NJ After 3 program. 



Exhibit 21 

Support and Resources Received from Externai Organizations, 2005-06 



Types of aid that are provided by at least one external organization... 



Special 

programs/activities/services are 
provided for youth on- or off-site 

Funding through grants or 
contracts is provided 



Students are referred to the NJ 
After 3 after-school program 



Donations of facilities or space 



T ranspo rtatio n services are 
provided forstudents 




0 20 40 60 80 100 



Percent of site coordinator survey respondents 



n Site coordinators (n=55) 



Exhibit reads: Seventy-six percent of site coordinators reported that at least one external 
organization provided special programs, activities, or services for youth either on- or off-site. 



Communications with the Schoois 

In survey responses, site coordinators identified topics and issues that they 
discussed with school principals or other school-day staff on at least a monthly 
basis, as shown in Exhibit 22. The sharing of classroom space and homework 
assignments were cited by coordinators as the topics most often discussed with 
school staff. Other discussion topics included student needs or progress, 
discipline policies, planning of after-school content, and after-school staffing. 
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Exhibit 22 

Program Communication with the Schooi, 2005-06 



Topics that are discussed with schooi staff at least once a month... 



Issues related to 
classrooms/sharing space 










Homework assignments 




The needs or progress of 










Student discipline policies 





Planning after-school program -71- I 

After-school program staffing 

0 20 40 60 80 100 

Percent of site coordinator survey respondents 

■ Site coordinators (n=55) 



Exhibit reads: Eighty-three percent of site coordinators reported that they discussed issues reiated 
to ciassrooms and the sharing of space with schooi staff at least once a month. 



The quality of the relationships between individual programs and schools 
varied considerably, with school facilities (as noted in survey responses) a 
particularly problematic issue in some sites. For example, at over half of the 
programs that we visited, staff described moderate to poor relationships with 
school-day teachers. Much of the reported tension surrounded the use of shared 
space and materials. “We have to beg, borrow, or steal from the [school-day] 
teachers if [we] want to show a video, or use a computer. . . we are not supposed to 
touch [their] stuff. Not even the pencil sharpener.” Another staff member 
described how school-day teachers at her site “leave little notes saying, ‘Don’t 
touch this or that. . . .’” As a result, many after-school staff members said that they 
had concluded that the school-day teachers lack respect for the after-school 
program. 

However, relationships between school-day staff and after-school staff 
were not poor in all program locations. Some after-school staff told us that they 
frequently communicate with school-day staff to learn what students are doing in 
the classroom, so that they can provide more targeted assistance or ensure that all 
homework is completed. “I communicate with a lot of teachers,” explained one 
after-school staff member. “For instance, [when] students tell me that they don’t 
have homework or that they have been having trouble. . .and don’t understand 
their work.” 
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Several site coordinators described efforts to align their program content 
with the school-day program. As one site director explained it, “We wrap around 
[the school] as much as possible. The principal tells us areas that are weak or 
when there are problems. We [also] support the holidays and [provide] extra 
activities [for students] .” 

Nevertheless, efforts to align the school-day and after-school programs 
are, in the view of some site coordinators and staff, sometimes hampered by 
philosophical differences between the organizations. One site coordinator 
summarized the problem this way: “The education system is too geared to 
standardized testing, and they don’t recognize that kids learn in other ways. The 
school staff don’t see the value of what we offer because it does not directly relate 
to standardized tests.” He emphasized his point by pointing to the fact that the 
school decided to extend instructional time and to reduce the time available for 
play during the school day. 



5. Baseline Information on Intended Program 
Outcomes 

This section presents baseline data in two areas in which program 
outcomes are being assessed, after-school availability and program sustainability. 



Availability of Services 

Parents at each of the locations we visited reported that the program 
offered through NJ After 3 was their only after-school option because it was free, 
conveniently located, and safe. In fact, all of the parents with whom we spoke 
said that there were no other suitable after-school options in their community that 
satisfied their dual need for safety and affordability. For example, when asked 
why she chose to enroll her child in the program, one parent said: “There are no 
other options for aftercare. I like that this is housed in [the school my child 
attends]. I don’t have to worry about my son getting on a bus and going 
elsewhere.” In response to the same question, another parent had a similar 
answer. “This program is so convenient. It’s safe, it’s educational, it’s fun. lean 
do my job without worrying.” Prior to this option becoming available, parents 
reported that they had taken their children to grandparents’ homes, hired 
babysitters, or, in a few cases, brought them to work. “Last year my child came to 
my job, and it was a drag for her,” explained one parent. 

Students also indicated that the NJ After 3 programming filled a vacuum 
that would not otherwise be filled with safe, positive, meaningful activities. 

When asked to describe what they might be doing if they were not in the after- 
school program, students frequently indicated that they would be at home 
sleeping, watching television or playing video games, or “getting into trouble.” 
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While a few students did indicate that they would likely attend sports classes or 
practices, most emphasized that the after-school program was the best option of 
those available to them. One student reflected this common sentiment when he 
told us: “[The program] helps us and makes us have fun. When we are not here, I 
[am] bored. . .flipping channels on my TV.” 

Site coordinators’ survey responses to questions on program availability 
supported these claims. Thirty-eight percent of the site coordinators surveyed 
confirmed that, prior to the NJ After 3 grant, there had been no after- school 
program offered by their organization at the current location. 

Several parents also indicated that the need for care extends beyond the 
school year to the summer months. One mother summed up the situation this 
way: 



“Last summer my daughter had to sit in front of the TV. Parents would 
really like to have this program during the summer. We lose our children 
to the street, it’s really infested with the gangs and that starts at this age. 
They really have nothing to do except go out on the streets. . . . While you 
are at work, you wonder where they are, what they are doing, who they 
are with. ’’ 

Survey results suggest, however, that very few (8 percent) executive 
directors of provider organizations are focused on developing summer 
programming. The majority of them have sought to take advantage of additional 
opportunities to expand and improve their existing after-school programs during 
the school year. For example, since receiving the NJ After 3 grant, 24 percent (7 
of 27) of executive directors reported increased opportunities to partner with a 
public school. A smaller number (4 of 27, or 16 percent) reported that the grant 
has increased opportunities to partner with cultural organizations. 

Although the NJ After grants resulted in some increase in opportunities for 
provider organizations, one anticipated benefit, which is the leveraging of 
corporate, private, or municipal funds, has not been realized at significant levels. 
In fact, between 41 and 70 percent of executive directors reported that the NJ 
After 3 grant had not affected their ability to leverage funds from each of these 
sources “at all.” For example, 41 percent of executive directors reported that their 
ability to leverage funds from private sources had not been affected at all by the 
grant, while 70 percent said the same regarding municipal sources. 

During our site visits, site coordinators provided examples that illustrated 
benefits that have been realized by NJ After 3 grantees. Among grantees that had 
operated programs before the NJ After 3 grant, coordinators noted that the NJ 
After 3 grant had increased availability of program resources, variety in program 
activities, training opportunities for staff, and capacity to serve more students. 

For example, staff at one program told us that, prior to receiving the grant, they 
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had only been able to provide students with homework assistance. Now that they 
have the grant, they are able to offer “tremendous [new] services” to kids in “a 
safe place, after hours.” Additional feedback on benefits and opportunities from 
site coordinators included the following: 

“Becoming a NJ After 3 site has helped us to get more space in our 
buildings and allowed us to expand our programs. Because of our grant, 
we can pay teachers to do the sports program and computers, and it has 
allowed us to raise the salary rate. . . ” 

“We expanded activities. [Where] else are these kids going to learn 
digital film editing or web design or martial arts? All of these 
extracurricular classes [are available to them even though their] parents 
can ’t afford to pay. . . ” 

“NJ After 3 has held a lot of trainings. . . We meet with other staff to 
network and exchange ideas. ” 

“[Before the grant] there was no K-3 program at this school. If we get 
more space, we will try to serve even more kids. ” 



Programmatic and Financial Sustainability 

The majority (34 of 55, or 62 percent) of site coordinators reported that they 
were in operation at their same location prior to receiving the grant award from NJ 
After 3, as shown in Exhibit 23. In fact, 22 percent (12 out of 55) reported that they 
had provided after-school programming at the site for more than five years. Among 
executive directors, about one-third (9 of 27) said that their organizations had been 
providing after-school programs for more than 25 years. 

Exhibit 23 

Number of Years That After-school Services Were Offered 
Prior to NJ After 3 Grant, 2005-06 



Number of Years 


Percent of Site Coordinators 

(n=55 ) 


After-school programming was not offered 
prior to the NJ After 3 grant 


38 


1-2 years 


33 


3-5 years 


7 


6-10 years 


6 


More than 10 years 


16 



Exhibit reads: Thirty-eight percent of responding site coordinators reported that after-schooi 
programming was not offered prior to the NJ After 3 grant at their program site. 
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According to the executive directors, on average, approximately 73 
percent of each site’s budget was provided by NJ After 3. The remaining 27 
percent came from general organizational funds (6 percent), funds from other 
state sources (6 percent), fees charged to families (4 percent), fees from other 
municipal sources (3 percent), federal funding sources (2 percent), and other 
organizations (7 percent). 



6. Baseline Information on Intended Participant 
Outcomes 

To achieve positive life outcomes, children and youth require 
opportunities and supports in multiple developmental domains, including 
academic, social, psychological, and physical fitness and health areas. This 
section describes baseline evidence of program-related outcomes in these areas. 



Opportunities for Exposure to New Opportunities 

More than 75 percent of student participants who were surveyed indicated 
that the NJ After 3 program had given them a chance to do a lot of new things, 
work on tasks that really made them think, and participate in activities that really 
held their interest, as shown in Exhibit 24. At least half (50 percent or more) 
agreed or strongly agreed that the program allowed them to do things that they 
usually did not get to do elsewhere, had a lot of activities to choose from, and had 
provided an opportunity for them to get involved in community service. 

Younger students in grades 3-5 reported more often than older ones in 
grades 6-8 that they agreed or strongly agreed with each of the questions that 
assessed students’ program experiences. Differences between younger and older 
students were particularly significant for some response categories. For example, 
whereas 86 percent of students in grades 3-5 agreed or strongly agreed that they 
got a chance to do a lot of new things in the program, 78 percent of students in 
grades 6-8 agreed or strongly agreed with the statement. Similarly, 82 percent of 
the younger students agreed or strongly agreed that the activities really got them 
interested, compared to 71 percent of the older students. These differences are 
typical of age-related differences found in other after- school programs. 
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Exhibit 24 

Youth Reactions Regarding Their Program Experience, Grades 3-8, 2005-06 

In this after-school program... 

I get a chance to do a lot of new things 
The activities really get me interested 
I get to work on activities that really make me think 
I get to do things that I don't usually get to do anyw here else 
There is a lot for me to choose to do 
I get to do community service projects 

0 20 40 60 80 100 

Percent of participants who reported they "agree" or "agree a lot" 

■ All responding participants (n=647) | 

Exhibit reads: Across grades 3-8, 84 percent of responding participants agreed or agreed a lot 
that in the after-school program they got a chance to do a lot of new things. 




Heip with Schooi 

The majority of students agreed or strongly agreed with several statements 
that were designed to assess their perception of whether the program had helped 
them with school, as shown in Exhibit 25. The largest proportion of students 
noted that the program had helped them to finish their homework more often (86 
percent) and to feel better about their schoolwork (78 percent). A similar 
proportion (77 percent) also said they felt the program had helped them to 
improve their grades. 
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Exhibit 25 

Students’ Perceptions of the Program’s Heip with Schooi, 
Grades 3-8, 2005-06 



This after-school program has helped me to... 







Finish my homework more often 








Feel better about my schoolw ork 








Get better grades in school 








Solve math problems better 








Read and understand better 








Write better 








Use computers to do schoolw ork better 





0 20 40 60 80 100 

Percent of participants who reported they "agree" or "agree a lot" 



n All responding participants (n=642) 



Exhibit reads: Across grades 3-8, 86 percent of responding participants agreed or agreed a lot that 
the after-school program had helped them to finish homework more often. 



The evaluation developed an academic-benefits scale based on the items 
that measured students’ reported academic benefits. Students who attended the 
program at the highly active level (80 percent of possible days and a minimum of 
80 days during the academic year) scored significantly higher on the academic 
benefits scale than their peers who did not attend as often. That means that the 
highly active participants were significantly more likely to agree or agree strongly 
with the statements regarding academic benefits of the program than were less 
active participants. On average, low-income students (i.e., those who were 
eligible to receive free or reduced-price lunches) were also significantly more 
likely than other students to agree or strongly agree with statements about the 
academic benefits of the program. This finding may reflect home and community 
situations in which low-income students have less access to the supports and 
opportunities available in the NJ After 3 program, compared to other students. 
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Sense of Attachment and Belonging 

Over three-quarters of all students surveyed agreed or agreed strongly with 
each of the statements that were designed to assess their sense of attachment and 
belonging to the NJ After 3 program, as shown in Exhibit 26. Ninety-one percent 
indicated that they felt safe, and almost as many indicated they felt like they 
belonged (87 percent) and that they felt successful (86 percent). Eighty-three 
percent felt “it’s a good place to hang out.” 



Exhibit 26 

Students’ Level of Attachment, Grades 3-8, 2005-06 



In this after-school program, I 

I am safe 
I belong 
I am successful 
I matter 

This Is a good place to hang out 
I have the pow er to help others 
My actions make a difference 
My Ideas count 

0 20 40 60 80 100 

Percent of participants who reported they "agree" or "agree a lot" 



n All responding participants (n=627) 

Exhibit reads: Across grades 3-8, 91 percent of responding participants agreed or agreed a lot 
that they felt safe in the program. 



feel like... 
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The parents with whom we spoke identified numerous ways in which they 
too believed their children had benefited through participation in the NJ After 3 
programs. In some cases, parents were able to quantify the academic progress 
that their children had made in terms of specific grades their children had earned, 
but often parents spoke of attitudinal and behavioral improvements such as higher 
levels of homework completion and independent reading. Some of the answers 
we received in response to questions regarding student outcomes are presented 
below. 
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“She is now an honor roll student. She was borderline before, now she is 
on the honor roll in language arts and mathematics. All of her grades 
have gone up. I also notice that she appears to seem to want to read 
more. She is carrying books every day [and] she orders books from all of 
the book clubs. She never did that before. ” 

“He actually likes to read now. It is hard for him to read the words, but 
the difference now is that he is not pushing the book aside. He wants to 
try to read. ” 

“The aftercare program gets the kids to focus. My child is happy to get 
the homework all done before he gets home. ” 

When pressed to identify what it is about the after-school program that 
accounts for academic improvement among their children, parents often pointed 
to the fact that time is expressly reserved for completing homework. In addition, 
several parents noted that the staff convey a willingness to help, and this in turn 
encourages their children to seek the academic assistance they need. 

The students whom we interviewed also discussed some of the ways in 
which they believe they have benefited academically through their participation in 
the program. Some were very specific: “My math has gotten better,” said one 
boy. “They showed me the right way to multiply and divide fractions.” Another 
told us her grades had improved: “[I get] all As and Bs.” Most were less specific, 
simply pointing out that “school is easy now” or that “tutoring helps.” Another 
informed us that he was in the chess club because “chess is a mind game and it 
helps you think better in school.” 



Academic Engagement and Skills 

Reading/language arts or homeroom teachers in the 10 NJ After 3 
programs in the in-depth sample completed a brief report on each participant in 
grades 3-6. The teacher survey data describe the academic characteristics of these 
students. The first year’s data will serve as a baseline to assess change in 
participants’ academic characteristics over the three-year longitudinal study. 

The survey asked teachers to assess students who were NJ After 3 
participants on the academic skills and other skills necessary for academic 
competence, including classroom participation, academic motivation, 
interpersonal skills, and study skills. Teachers based their assessments on the 
skills and behaviors that after-school participants exhibited during the regular 
school day. 
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According to the responding teachers: 

■ 75 percent of after-school participants “almost always” or “often” 
demonstrated academic engagement by speaking in class when 
they were called upon. Almost 60 percent of participants 
demonstrated six out of eight listed behaviors that relate to 
academic engagement. (Academic engagement indicates students’ 
willingness to volunteer to answer questions and readiness to 
participate in class discussions.) 

■ Less than half (33 to 49 percent) of after-school participants almost 
always or often demonstrated 10 out of 1 1 listed behaviors of 
academic motivation. (Academic motivation indicates students’ 
eagerness to learn, willingness to take on challenges, ability to stay 
on target, and evidence of a sense of responsibility for their own 
learning.) 

■ Well over half (59 to 69 percent) of after-school participants 
almost always or often demonstrated all 10 interpersonal skills. 
(Interpersonal skills indicate how well students follow rules, accept 
limits, and interact with adults and their peers.) 

■ Over half of after-school participants almost always or often 
demonstrated 10 of the 11 study skills. (Study skills illustrate how 
well students prepare for tests, how often they complete 
homework, and how often they correct their own work.) 

■ Well over half (64 to 75 percent) of after-school participants 
demonstrated good or excellent skills in using computers for four 
out of five tasks. (Technology skills refer to students’ ability to 
use word processing programs, use the Internet for research, and 
send and receive e-mails.) Students fared the lowest (46 percent) 
on using spreadsheet programs, such as Excel. 

■ In comparison with grade-level expectations at the school, well 
over half (64 to 73 percent) of after-school participants across 
grades 3-6 met grade-level expectations on all 1 1 reading and 
language arts skills. (Reading and language arts skills denote 
students’ ability to identify a main idea, use grammar and 
punctuation correctly, and draw conclusions from written 
material.) 

Further analysis revealed significant differences in teacher assessments 
between low-income students and other students, and between younger students 
in grades 3-4 and older students in grades 5-6. In general, analysis revealed that 
in many skill and knowledge areas, fewer low-income student participants were 
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rated as possessing the skills that teaehers deem desirable for aeademie sueeess, 
when compared to their higher-income peers who were also enrolled in the after- 
school program. The same pattern held true for younger versus older students, 
with fewer younger students rated as possessing needed skills. This suggests that 
after- school participants from poor families may have the greatest need for the 
skill-building and academic activities that are offered after school by NJ After 3. 



Social Development and Enrichment 

Although academic benefits were important to all concerned, many 
parents were equally pleased with the social benefits that their children had 
realized through their participation in various enrichment activities. “They 
provide so many different activities,” said one parent. “Sewing, modeling, arts 
and crafts, music, [and] of course homework help is number one, first and 
foremost.” Others expressed appreciation for the field trips. “I like that they go 
on trips,” said one parent. “My health is not great, so this. . .is the only way [my 
child] would get to go. I like that the children are exposed to elements that are 
outside of [our community] . . .museums, educational programs, things like that.” 

Parents shared several examples of incidents and situations that highlight 
the ways in which their children have developed in social or non-academic ways. 
They included the following: 

“The other day someone must have talked to [my son] about being 
healthy, ” reported one mother. “So, he comes home and says, ‘Mom, we 
gotta get rid of some of this junk food. ’ That tells me he’s been paying 
attention. He [also] loves computers. It helps him work on his fine motor 
skills. Cooking class... it’s all about listening. [He has] improved 
listening skills. ’’ 

“I remember when [my daughter] said that she had no friends, and now 
she does, and she has a good rapport with the staff. They are like 
family — they really treat them that way. I see improvement as far as her 
self-esteem. Before she was more of an introvert. ..and did not have 
friends. Now she is more outgoing and has more friends. . . ’’ 



Healthful Living 

Over the past three decades, the rate of obesity among children has 
doubled, from 15 percent to nearly 30 percent today.'* According to research 
experts who convened at the Forum on Childhood Obesity in early 2006, many 



Anderson, P., & Butcher, K. (2006). Childhood obesity: Trends and potential causes. The 
Future of Children: Childhood Obesity, 16(1), pp. 19-46. 
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factors contribute to the growing problem of childhood obesity. An increasing 
number of studies show that childhood obesity can be improved by interventions 
in schools that engage children and their families in healthful activity and proper 
nutrition. In response, NJ After 3 has adopted the promotion of healthy 
development among children as one of its goals. 

The participant survey data describe the indicators of healthy lifestyles 
exhibited by students in grades 3-8 who participate in NJ After 3. These first-year 
data will serve as baseline for tracking participants’ health characteristics during 
the three-year longitudinal study. 

In addition to participation in physical activities as reported in Exhibit 1 1 
and accompanying text, participants answered questions about their nutrition and 
sleeping habits. Irrespective of grade or attendance level, participants were most 
likely to report that they ate fast food from a fast- food restaurant one to two times 
a week, as shown in Exhibit 27. A quarter more participants ate fast food at least 
three times a week or more. These data on eating habits confirm the need for 
activities that educate participants on nutrition. 



Exhibit 27 

Eating Habits of Students by Grade Ciuster, Grades 3-8, 2005-06 



Number of times a week students eat 
fast food from a fast food restaurant 
(n=643) 


Percent of Students 


Grades 3-5 

(n=472) 


Grade 6-8 

(n=188) 


All Grades 

(n=643) 


Less than once a week 


28 


23 


27 


One to two times a week 


50 


51 


48 


Three times a week or more 


23 


26 


24 



Exhibit reads: Across grades 3-8, 27 percent of participants ate fast food from a fast food 
restaurant iess than once a week. Twenty-eight percent of partioipants in grades 3-5 reported as 
suoh, compared to 23 percent of students in grades 6-8. 



Participants in grades 3-5 were more likely to report that they slept at least 
eight hours on a typical night, compared to older students in grades 6-8, as shown 
in Exhibit 28. Eor example, while 77 percent of students in grades 3-5 (343 of 
446) reported that they slept on a school night at least eight hours, only 62 percent 
of youth in grades 6-8 (1 14 of 184) reported that they did the same. Similarly, 64 
percent of youth in grades 3-5 (266 of 415) reported that they slept on a weekend 
night at least eight hours, compared to 55 percent of youth in grades 6-8 (98 of 
178). These data suggest that about a quarter to almost half of students in all 
grades were not getting the minimum amount of sleep required for optimum 
concentration and mental acuity in school. 



41 



















Exhibit 28 

Amount of Sieep Students Reported by Grade Ciuster, Grades 3-8, 

2005-06 



1 get at least 8 hours of sleep 
on a typical... 


Percent of Students 


Grades 3-5 

(n=446) 


Grade 6-8 

(n=184) 


All Grades 

(n=636) 


School night (n=630)* 


77 


62 


72 


Weekend night (n=593) 


64 


55 


61 



* These differences were statistically significant (p<.05). 

Exhibit reads: Across grades 3-8, 72 percent of participants reported that they got at least eight 
hours of sleep on a typical school night. Seventy-seven percent of participants in grades 3-5 
reported that much sleep, compared to 62 percent of students in grades 6-8. 



Other Reactions of Famiiies 

Many parents expressed strong feelings of relief and gratitude that the NJ 
After 3 program was available to their children. In addition, some were eager to 
describe how they and their families had benefited through their children’s 
participation. “My girls are timid [but] since starting this program they are more 
outgoing. . .1 like that [program staff] check homework here, too, so I don’t have 
to be behind them about it when I get home,” said one mother. Others 
emphasized safety. “I don’t let [my children] hang outside, so [this program] is 
their only chance to leave the house after school. Safety is a big issue here, the 
streets are very bad and kids do not go out to play like they want to. They don’t 
get to play after school. . .to be safe they have to stay in the house.” Other parents 
expressed similar sentiments. “I don’t want [my child] playing outside where we 
live, there are too many negative people and I will not let her play with them,” 
said one mother. Another reiterated: “There is an element of danger that exists [in 
my neighborhood].” 

In addition to experiencing peace of mind, parents also reported practical 
economic and personal benefits, such as the ability to focus on their jobs and the 
freedom and flexibility to attend to personal responsibilities. For example, one 
mother said she was able to take her younger child, who is chronically ill, to 
medical appointments without having to subject her older child to this time- 
consuming and boring activity. Other parental feedback included the following: 

“I am happy to know where she is... in a structured environment from 3-6 
PM. It gives me peace of mind. ” 

“There are things I have to do at work. Meetings. I know the program 
closes at 6 PM so I can stay longer [at work] because I know that my child 
is in a secure place. It’s helped our family. ” 
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“I love it. It does help me. 1 was a housemother for 7 years, but now I am 
a nurse. I wanted to work eight hours [and] now I can do that. ” 



7. Findings About Important Program Features 

This final section identifies those program structures and practices that 
have the potential to most strongly affect the achievement of desirable program 
goals. The discussion is framed by key quality indicators that researchers in the 
field of youth development have determined can promote positive changes among 
students who participate in out-of-school time programs. In addition, we examine 
those aspects of the NJ After 3 programs that appear to be most responsive to 
parent and student needs and that also enhance the quality of the experience for 
participants. 



Availability of Rich Content-based Activities 

Students and their parents want after-school programs to provide 
information and learning opportunities that are not available through their schools 
and that family resources cannot provide. Programs recognized this need and 
have tried to expand offerings and hire appropriate staff for the clubs and 
activities that students and parents have requested. The need is great, however, 
and programs confront obstacles such as limited space, unqualified staff, or 
inadequate financial resources as they try to introduce new classes and activities. 

Although students voiced frustration and criticism infrequently, when they 
did, they typically spoke of their inability to enroll in the clubs or activities that 
they considered to be most interesting. “I signed up for advanced dance, but I got 
placed in sewing,” lamented one of many students who expressed frustration over 
their activity placement. Some programs have tried to address this issue by 
actively soliciting ideas from students about the activities that interest them most. 
The results have been generally positive. For example, one student explained that 
a staff member “took notes about what we want to do that would be more helpful 
or more fun. [As a result] we got math games, and science games, and we have 
more time to play leap tag. . . .” 

The need to keep students challenged and engaged through the provision 
of rich content-based activities is particularly important for retaining older 
students. The participant survey data indicate that, in general, fewer of those in 
grades 6-8 experienced the same high level of satisfaction with program offerings, 
compared to their younger peers. In addition, older students attended the after- 
school programs with less regularity than did younger students. Although very 
few (12 percent) site coordinators considered enrollment and attendance levels to 
present challenges, retention was lower among older students. This issue may be 
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able to be addressed through program content that meets adolescents’ need for 
challenge, variety, and interest. 

One programming area that could be further developed is community 
service outside the school setting. Site-visit interviews with 10 site coordinators 
and 63 students indicated that, within this group of 10 programs, the availability 
of service learning projects was limited. Fewer than half of the site coordinators 
(45 percent) in the program overall reported that their programs offered service 
projects outside the school. Students expressed great enthusiasm about such 
activities and wanted more. “I wish we could help out senior citizens or 
something, but we don’t,” said one student. Another student told us she planned 
to bring this issue to the attention of the site coordinator and request that 
opportunities for service be explored. We also heard requests for more field trips, 
conflict resolution programs, health and nutrition activities, hands-on science 
activities, technology, dance, and drama. 



Delivery of Learning- and Mastery-oriented Content 

Students were particularly engaged by activities that were structured and 
focused on the achievement of clear goals. Long-term projects in dance, drama, 
and fashion typically satisfied these criteria and were immensely popular among 
students. In these classes, students honed their skills and routines over a period of 
weeks in preparation for a culminating event. Similarly, when students made 
kitchen utensils in a woodwork class, participated in a chess tournament, or sewed 
an outfit, the process of learning and developing mastery was evident to them and 
to others. However, guiding students through the learning process associated with 
special projects takes time and concentrated effort. Sufficient time must be 
allotted for these activities to minimize disruptions that interrupt the learning 
experience, and after-school program schedules should be developed with this 
goal in mind. 

Requiring staff to develop projects or activities with long-term goals and 
objectives may not always be feasible given limitations related to staff expertise, 
time, and other resources. Nevertheless, staff should be encouraged to develop 
programs that allow students to experience growth and progress over time and to 
demonstrate that progress to others. As new activities are introduced, those that 
offer the most potential for learning and mastery could be given highest priority. 

Fifty-nine percent of site coordinators reported that they used published or 
externally-developed curricula to guide some of the activities that were offered at 
their programs. In most cases, program staff designed their own lesson plans and 
developed the content and activities for each session. While this allowed 
instructors to be flexible and responsive to student interests and needs, it can also 
have the unintended outcome of uneven quality within and across programs. This 
possibility is increased by the fact that only two-thirds (67 percent) of site 
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coordinators required most or all of their staff to submit lesson plans for review 
on a regular basis. As the goal of an activity shifts from content exposure to a 
deeper, perhaps longer-term learning experience, staff will need to be even more 
intentional about planning and curriculum design. 



Practices to Support Positive Reiationships 

NJ After 3 programs facilitated the development of positive relationships 
between staff and students and among students at many sites. Through both 
formal activities and informal interactions, students and staff got to know each 
other and reported relationships of trust and caring. Site visit observations and 
survey data suggest that, in general, students were comfortable and relaxed in 
their interactions with their peers and with the adults in the program. 

The significant presence of youthful staff members with whom students 
feel they can relate and the programs’ low student staff-ratios supported these 
positive relationships. Almost all of the site coordinators (91 percent) indicated 
that they agreed or strongly agreed that student groups within the program were 
small enough for staff to meet participants’ individual needs. 

Relationships also developed within activities that encouraged 
communication between staff and students during tasks that engaged students 
physically and mentally. Examples of such activities included a dance class in 
which students and staff choreographed dance moves together, or a character- 
development class during which staff required students to analyze, defend, and 
discuss their opinions and beliefs. “Structured” free time during which students 
could set the agenda but which required staff to actively engage with participants 
on a one-on-one basis or in small groups also promoted positive relationships. 

NJ After 3 should work to keep student-staff ratios low and encourage 
staff to reach out to students to initiate new relationships and build trust. 
Similarly, staff should be intentional in their organization and planning of games 
and activities that require students to communicate with and assist each other. 



Strong Partnerships and Links with Famiiies, Schoois, and Other 
Organizations 

Staff at after-school programs expressed awareness of the multiple needs 
that existed among program participants. These needs were academic, social, 
physical, and emotional. Staff acknowledged that it was unreasonable to expect a 
single organization to adequately address all of the needs. However, because the 
organizations and staff members who provide after-school services had expertise, 
experience, and personal contacts in the fields of both social services and 
education, they were uniquely positioned to identify resources and information for 
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program participants and their families. The result was that many after-school 
programs served as conduits between families and community resources, helping 
to identify needs, raise awareness, and provide necessary services. 

Parents also received information from the after-school program regarding 
program-related activities and upcoming events, and about their children’s 
behavior and achievements. However, much of the interaction between staff and 
parents occurred informally, such as during pick-up at the end of the program day 
or on the telephone during evenings and weekends, when a specific issue arises. 
Teacher survey data revealed that the amount of communication varied 
significantly among parents. Teachers reported that they communicated with 
African American and white parents more often than they did with Latino and 
Asian parents. 

As the number of students from Latino and Asian ethnic groups, especially 
recent immigrants, grows in many communities, after-school programs need to 
become more responsive to these populations. While it is not entirely clear from 
our research that language differences explain this finding, it may indeed be a 
contributing factor. Sixty-two percent of site coordinators reported that too few 
staff with skills to work with English Language Learners was a major challenge. 

Program relationships with schools also varied by location. Some sites 
were still trying to develop procedures and establish boundaries and expectations 
with their host schools. All of the site coordinators with whom we spoke said that 
sharing space and materials was a challenge. According to one site coordinator, 
“The school is not big enough; other programs are here. We can’t have a whole 
section of the school.” Others had good working relationships, and it was likely 
that students benefited from the efforts of school staff and after-school staff to 
share ideas and materials, and to reinforce what each other was doing to promote 
academic achievement. As one site coordinator said, she would like her program 
to be “somewhat consistent with the day school but different enough to keep them 
interested and keep them coming.” As we heard in several programs, the high 
rates of staff turnover in many schools will challenge these efforts. “Lots of new 
teachers were hired at the school this year and they are not used to sharing their 
space. If [the after-school students or staff] leave one little thing out of place, 
they blow it up way out of proportion,” said one site coordinator. The arrival of 
each new administrator, and to some extent classroom teacher, signals the need to 
build new bridges and begin new conversations regarding responsibilities and 
expectations. 
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Looking Toward Year 2 

This report describes what we have learned about the program goals, 
participants, staff, activities, operations, and short-term outcomes of this initiative 
based on data collected from Year 1 of the evaluation. These descriptions of 
programs and students in 2005-06 will serve as baseline data for the longitudinal 
analyses that will be conducted in Years 2 and 3 of the evaluation. Future phases 
of the evaluation will be able to provide more information on the extent to which 
programs and participants were able to meet the initiative’s goals. 
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Technical Appendix 




Method Used in Selecting the In-depth Study Sample 



The evaluation employed a nested sample to collect data from all NJ After 3 
projects. A graphic representation of this sampling procedure is provided in Exhibit A-1. 
The sampling design permits us to collect certain types of overview data from all 
program sites, with progressively more detailed information available from smaller 
samples. The nested sampling strategy also permits us to make informed estimates of 
overall program and outcome patterns based on information obtained from subsets of 
programs and participants. 

To select the ten programs for the in-depth evaluation sample, we conducted a 
random selection from the pool of Round 1 programs. We used this selection method 
because the number of possible sites (21) was too small to extract a statistically 
significant sample based on stratification variables. The random assignment yielded a 
sample of elementary and middle school programs located in six out of the nine counties 
represented in the NJ After 3 initiative. 



Exhibit A-1 

NJ After 3 Evaiuation Nested Sampie Approach 




A-1 




Details of Data Used in Analyses of Baseline Participant 
Characteristics, Based on Participant Survey 



Exhibit A-2 

Participant Seif-Reports of Physicai Activity, by Grade Levei 



Percent of participants who reported engaging in the activity at ieast three hours a week 


In a given week, 1... 


Grades 3-5 (n=445) 


Grades 6-8 (n=183) 


Chi-Square 


P 


Play basketball, football, or soccer 


45 


37 


3.30 


0.08 


Run or walk 


42 


35 


2.91 


0.10 


Participate in physical activity that 
isn’t listed above* 


37 


28 


5.06 


0.03 


Dance as part of a group* 


29 


17 


9.19 


0.00 


Jump Double Dutch* 


23 


15 


4.41 


0.04 


Lift weights* 


21 


12 


7.06 


0.01 


Do martial arts like Taekwondo or 
judo* 


21 


6 


20.22 


0.00 



* These differences were statistically significant (p < .05). 



Exhibit reads: Forty-five percent of students in grades 3-5 reported that in a given week they played 
basketball, football, or soccer at least three hours, compared to 37 percent of students in grades 6-8. This 
difference was not statistically significant. 



Exhibit A-3 

Participant Seif-Reports of Engagement in Inactive Behaviors, by Grade Levei 



Percent of participants who reported inactive behavior four or more hours per school day 


On a given school day, 1... 


Grades 3-5 (n=435) 


Grades 6-8 (n=181) 


Chi-Square 


Q. 


Watch television 


39 


42 


0.50 


0.52 


Play video games 


23 


17 


2.25 


0.16 



Exhibit reads: Thirty-nine percent of students in grades 3-5 reported that on a given school day they watched 
television for four or more hours, compared to 42 percent of students in grades 6-8. This difference was not 
statistically significant. 
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Details of Data Used in Analyses of Participant Experiences 
and Program Features 

Participant Survey Responses 

Exhibit A-4 

Participant Seif-Reports of Exposure to New Opportunities, by Grade Levei 



Percent of participants who “agreed” or “agreed a iot” to exposure to new opportunities 


in this program... 


Grades 3-5 (n=454) 


Grades 6-8 (n=188) 


Chi-Square 


P 


1 get a chance to do a lot of new 
things* 


86 


78 


6.72 


0.01 


1 get to work on activities that really 
make me think* 


82 


71 


9.19 


0.00 


The activities really get me 
interested 


80 


75 


2.02 


0.17 


1 get to do things that 1 don’t usually 
get to do anywhere else* 


76 


67 


5.01 


0.03 


There is a lot for me to choose to 
do 


71 


65 


2.32 


0.13 


1 get to do community service 
projects* 


53 


44 


4.34 


0.04 



* These differences were statistically significant (p < .05). 



Exhibit reads: Eighty-six percent of students in grades 3-5 “agreed” or “agreed a lot” that they got a chance to 
do a lot of new things in the pregram, compared to 78 percent of students in grades 6-8 who replied the same. 
This difference was statistically significant. 
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Participant Survey Scaies 



Exposure to New Opportunities 

The Exposure to New Opportunities scale was computed to range from one to four, with four 
indicating that on average participants strongly agreed with the following statements: 

In this afterschool program. . . 

■ I get a chance to do a lot of new things 

■ I get to do things that I don’t usually get to do anywhere else 

■ I get to work on activities that really make me think 

■ There is a lot for me to choose to do 

■ The activities really get me interested 

■ I get to do community service projects 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.79 


3.01 


.071 


1 


2.67 


3.50 


4 




1.0 

Disagree a lot 



1.5 



2.0 2.5 3.0 

Scale score 



3.5 4.0 

Agree a lot 
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Community Service Projects 

The Community Service Projects scale was computed to range from one to four, with four 
indicating that on average participants strongly agreed with the following statements: 

Participating in community service projects with this afterschool program, I feel like. . . 

■ My actions make a difference in the community 

■ My actions help others 

■ I have learned more about my community 

■ I have learned more about how I can help others 

■ I have learned more about other organizations in my community 

■ It is important to volunteer and help others 

■ I will continue to volunteer to help others in my community 

■ I can call myself a volunteer 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.92 


2.94 


0.89 


1 


2.50 


3.63 


4 
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Sense of Belonging 

The Sense of Belonging scale was computed to range from one to four, with four indicating that 
on average participants strongly agreed with the following statements: 

In this afterschool program, I feel like. . . 

■ I belong 

■ My ideas count 

■ lam successful 

■ This is a good place to hang out 

■ I matter 

■ lam safe 

■ My actions make a difference 

■ I have the power to help others 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.87 


3.35 


0.68 


1 


3.00 


4.00 


4 
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Interactions with Staff 

The Interactions with Staff scale was computed to range from one to four, with four indicating 
that on average participants strongly agreed with the following statements: 

In this afterschool program. . . 

■ Staff treat me with respect 

■ I feel that I can talk to staff about things that are bothering me 

■ Staff really care about me 

■ Staff often keep their promises 

■ Staff care what I think 

■ Staff try to be fair 

■ Staff think I can do things well 

■ Staff help me to try new things 

■ Staff think I can learn new things 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.91 


3.31 


0.71 


1 


2.89 


4.00 


4 




1.0 1.5 

Disagree a lot 



2.0 2.5 3.0 

Scale score 



3.5 4.0 

Agree a lot 
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Interactions with Peers 



The Interactions with Peers scale was computed to range from one to four, with four indicating 
that on average participants strongly agreed with the following statements: 

In this afterschool program, I. . . 

■ Get to know other kids really well 

■ Can really trust the other kids 

■ Have a lot of friends 

■ Like the other kids 

■ Have a good time playing with other kids 

■ Get along with other kids 

■ Get to work with other kids as part of a team 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.84 


3.27 


0.64 


1 


2.86 


3.86 


4 




1.0 

Disagree a lot 



1.5 



2.0 2.5 3.0 

Scale score 



3.5 



4.0 

Agree a loti 
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Details of Data Used in Analyses of Participant Academic 
Engagement and Skills 



Participant Survey Scaies 



Academic Benefits of the Program 

The Academic Benefits of the Program scale was computed to range from one to four, with four 
indicating that on average participants strongly agreed with the following statements: 

The afterschool program has helped me. . . 

■ Get better grades in school 

■ Feel better about my schoolwork 

■ Read and understand better 

■ Solve math problems better 

■ Finish my homework more often 

■ Write better 

■ Use computers to do schoolwork better 
Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.86 


3.03 


0.79 


1 


2.58 


3.71 


4 
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Teacher Survey Responses 



By Grade Level 



Exhibit A-5 

Teacher Reports of Participants’ Interpersonai Skiiis, by Grade Levei 



Percent of participants who “aimost aiways” or “often” exhibited the foiiowing interpersonai skiiis 




Grades 3-4 (n=161) 


Grades 5-6 (n=142) 


Chi-Square 


P 


Gets along with people who are 
different 


69 


68 


0.01 


1.00 


Follows classroom rules 


67 


67 


0.01 


1.00 


Corrects inappropriate behavior 
when asked 


67 


67 


0.00 


1.00 


Interacts appropriately with adults 


67 


66 


0.03 


0.90 


Works effectively in a small group 
activity 


66 


66 


0.00 


1.00 


Listens to what others have to say 


66 


61 


0.68 


0.47 


Interacts appropriately with other 
students 


65 


65 


0.01 


1.00 


Accepts suggestions from teachers 


65 


59 


1.18 


0.29 


Works effectively in a large group 
activity 


63 


62 


0.02 


0.91 


Expresses dissatisfaction 
appropriately 


55 


63 


2.37 


0.13 



Exhibit reads: According to teachers, 69 percent of students in grades 3-4 “aimost aiways” or “often” foliowed 
ciassroom ruies, compared to 68 percent of students in grades 5-6. This difference was not statisticaliy 
signifioant. 
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Exhibit A-6 

Teacher Reports of Participants’ Academic Engagement, by Grade Levei 



Percent of participants who “aimost aiways” or “often” exhibited 
the foilowing academic engagement behaviors 




Grades 3-4 (n=161) 


Grades 5-6 (n=142) 


Chi-Square 


P 


Speaks in class when called upon 


76 


75 


0.05 


0.89 


Participates in class discussions 


58 


64 


1.26 


0.29 


Volunteers answers to questions 


57 


63 


0.96 


0.35 


Volunteers to read aloud 


55 


63 


1.10 


0.16 


Asks questions about tests or 
projects 


55 


59 


0.62 


0.49 


Asks questions when confused 


55 


57 


0.10 


0.82 


Initiates conversations appropriately 


53 


60 


1.53 


0.25 


Assumes leadership in group 
situations 


38 


44 


1.31 


0.29 



Exhibit reads: According to teachers, 76 percent of students in grades 3-4 “aimost aiways” or “often” spoke in 
ciass when caiied upon, oompared to 75 percent of students in grades 5-6. This difference was not 
statisticaily signifioant. 



Exhibit A-7 

Teacher Reports of Participants’ Academic Motivation, by Grade Levei 



Percent of participants who “aimost aiways” or “often” exhibited 
the foiiowing academic motivation behaviors 




Grades 3-4 (n=161) 


Grades 5-6 (n=142) 


Chi-Square 


P 


Is motivated to learn 


59 


59 


0.01 


1.00 


Stays on task 


45 


53 


1.99 


0.17 


Makes the most of learning 
experiences 


44 


49 


0.61 


0.49 


Attempts to improve on previous 
performance 


41 


49 


2.10 


0.17 


Persists when task is difficult 


40 


44 


0.50 


0.49 


Assumes responsibility for own 
learning 


36 


44 


1.84 


0.20 


Prefers challenging tasks 


36 


43 


1.52 


0.24 


Is goal-oriented 


35 


42 


1.21 


0.29 


Produces high-quality work* 


32 


47 


6.97 


0.01 


Looks for ways to academically 
challenge self 


30 


38 


1.94 


0.18 


Critically evaluates own work* 


28 


39 


4.48 


0.04 



'These differences were statisticaiiy significant (p < .05). 



Exhibit reads: According to teachers, 59 percent of students in grades 3-4 “aimost aiways” or “often” were 
motivated to iearn, compared to 59 percent of students in grades 5-6. This reiationship was not statistioaiiy 
signifioant. 
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Exhibit A-8 

Teacher Reports of Participants’ Study Skiiis, by Grade Levei 



Percent of participants who “almost aiways” or “often” exhibited the following study skills 




Grades 3-4 (n=161) 


Grades 5-6 (n=142) 


Chi-Square 


Q. 


Completes homework 


72 


63 


3.03 


0.09 


Turns in homework on time 


69 


59 


3.61 


0.07 


Takes care of materials 


62 


70 


2.26 


0.15 


Completes assignments according 
to directions 


58 


64 


1.26 


0.29 


Pays attention in class 


58 


59 


0.06 


0.82 


Prepares for class 


58 


58 


0.01 


1.00 


Finishes class work on time 


56 


67 


3.84 


0.06 


Prepares for tests 


52 


56 


0.70 


0.42 


Corrects own work 


48 


51 


0.9 


0.57 


Takes notes in class 


47 


58 


3.77 


0.07 


Reviews materials 


44 


52 


2.26 


0.14 



Exhibit reads: According to teachers, 72 percent of students in grades 3-4 “aimost aiways” or “often” 
compieted homework, compared to 63 percent of students in grades 5-6. This difference was not statisticaiiy 
signifioant. 
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Exhibit A-9 

Teacher Reports of Participants’ Reading and Language Arts Skiiis, 

by Grade Levei 



Percent of participants who met at ieast grade-ievel expectations for 
the foiiowing reading and language arts skills 




Grades 3-4 (n=161) 


Grades 5-6 (n=141) 


Chi-Square 


P 


Oral communication 


70 


77 


2.29 


0.15 


Spelling* 


62 


75 


5.26 


0.03 


Reading comprehension 


62 


72 


3.46 


0.07 


Reading fluency 


62 


71 


2.47 


0.14 


Word-attack 


62 


70 


2.27 


0.15 


Vocabulary 


62 


69 


1.47 


0.28 


Punctuation 


61 


71 


2.97 


0.09 


Drawing conclusion from written 
material* 


60 


73 


6.03 


0.02 


Identifying a main idea* 


59 


73 


6.57 


0.01 


Grammar 


59 


70 


4.10 


0.05 


Written communication* 


58 


70 


4.37 


0.04 



'These differences were statistically significant (p < .05). 



Exhibit reads: According to teachers, 70 percent of students in grades 3-4 met at least grade-level 
expectations regarding reading comprehension skills, compared to 77 percent of students in grades 5-6. 
This difference was not statistically significant. 



Exhibit A-10 

Teacher Reports of Participants’ Technoiogy Skiiis, by Grade Levei 



Percent of participants who exhibited “excellent” or “good” technology skills 




Grades 3-4 (n=135) 


Grades 5-6 (n=124) 


Chi-Square 


P 


Playing games* 


68 


82 


6.66 


0.01 


Use of Internet for research* 


54 


73 


8.79 


0.00 


Use of word processing program* 


51 


77 


16.36 


0.00 


Sending and receiving e-mail* 


49 


80 


18.09 


0.00 


Use of spreadsheet program* 


26 


72 


29.32 


0.00 



'These differences were statistically significant (p < .05). 

Exhibit reads: According to teachers, 68 percent of students in grades 3-4 exhibited “excellent” or “good” 
technology skills, compared to 82 percent of students in grades 5-6. This difference was statistically 
significant. 
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By Free- or Reduced-Price Lunch Status 



Exhibit A-11 

Teacher Reports of Participants’ Interpersonai Skiiis, 
by Free- or Reduced-Price Lunch Status 



Percent of participants who “aimost aiways” or “often” exhibited the foiiowing interpersonai skiiis 




Received Free- or 
Reduced-Price 
Lunch (n=119) 


Did Not Receive 
Free- or Reduced- 
Price Lunch (n=58) 


Chi-Square 


P 


Gets along with people who are 
different 


70 


76 


0.72 


0.40 


Interacts appropriately with other 
students 


68 


76 


1.14 


0.29 


Interacts appropriately with adults 


66 


79 


3.52 


0.06 


Works effectively in a small group 
activity 


66 


76 


1.65 


0.20 


Follows classroom rules* 


63 


83 


7.16 


0.01 


Corrects inappropriate behavior 
when asked* 


62 


83 


7.71 


0.01 


Listens to what others have to say* 


62 


78 


4.20 


0.04 


Works effectively in a large group 
activity 


62 


76 


3.28 


0.07 


Expresses dissatisfaction 
appropriately 


60 


71 


2.04 


0.15 


Accepts suggestions from teachers* 


58 


74 


4.38 


0.04 



*These differences were statistically significant (p < .05). 



Exhibit reads: According to teachers, 70 percent of students who received free- or reduced-price lunch 
“almost always” or “often” followed classroom rules, compared to 76 percent of students who did not receive 
free- or reduced-price lunch. This difference was not statistically significant. 
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Exhibit A-12 

Teacher Reports of Participants’ Academic Engagement, 
by Free- or Reduced-Price Lunch Status 



Percent of participants who “aimost aiways” or “often” exhibited 
the foiiowing academic engagement behaviors 




Received Free- or 
Reduced-Price 
Lunch (n=119) 


Did Not Receive 
Free- or Reduced- 
Price Lunch (n=58) 


Chi-Square 


P 


Speaks in class when called upon 


79 


86 


1.34 


0.25 


Asks questions when confused 


61 


59 


0.12 


0.73 


Participates in class discussions 


60 


71 


2.04 


0.15 


Volunteers answers to questions 


59 


69 


1.71 


0.19 


Volunteers to read aloud 


58 


69 


1.99 


0.16 


Asks questions about tests or 
projects 


56 


71 


3.39 


0.07 


Initiates conversations appropriately 


56 


67 


1.94 


0.16 


Assumes leadership in group 
situations 


40 


53 


3.08 


0.08 



Exhibit reads: According to teachers, 79 percent of students who received free- or reduced-price iunch 
“aimost aiways” or “often” spoke in ciass when caiied upon, compared to 86 percent of students who did not 
receive free- or reduced-price iunch. This difference was not statisticaiiy significant. 
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Exhibit A-13 

Teacher Reports of Participants’ Academic Motivation, 
by Free- or Reduced-Price Lunch Status 



Percent of participants who “aimost aiways” or “often” exhibited 
the foiiowing academic motivation behaviors 




Received Free- or 
Reduced-Price 
Lunch (n=119) 


Did Not Receive 
Free- or Reduced- 
Price Lunch (n=58) 


Chi-Square 


P 


Is motivated to learn* 


53 


76 


8.57 


0.00 


Stays on task* 


45 


66 


6.87 


0.01 


Attempts to improve on previous 
performance 


45 


55 


1.77 


0.18 


Makes the most of learning 
experiences* 


40 


60 


6.82 


0.01 


Persists when task is difficult* 


38 


57 


5.76 


0.02 


Assumes responsibility for own 
learning* 


35 


59 


8.66 


0.00 


Prefers challenging tasks* 


35 


53 


5.83 


0.02 


Is goal-oriented* 


33 


59 


10.75 


0.00 


Produces high-quality work* 


33 


55 


8.14 


0.00 


Looks for ways to academically 
challenge self* 


29 


48 


6.65 


0.01 


Critically evaluates own work* 


29 


45 


4.60 


0.03 



'These differences were statistically significant (p < .05). 



Exhibit reads: According to teachers, 53 percent of students who received free- or reduced-prioe lunch 
“almost always” or “often” were motivated to learn, compared to 76 percent of students who did not receive 
free- or reduced-price lunch. This difference was statistically significant. 
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Exhibit A-14 

Teacher Reports of Participants’ Study Skiiis, 
by Free- or Reduced-Price Lunch Status 



Percent of participants who “almost aiways” or “often” exhibited the following study skills 




Received Free- or 
Reduced-Price 
Lunch (n=119) 


Did Not Receive 
Free- or Reduced- 
Price Lunch (n=58) 


Chi-Square 


P 


Completes homework 


66 


78 


2.67 


0.10 


Takes care of materials 


64 


76 


2.57 


0.11 


Finishes class work on time* 


62 


79 


5.24 


0.02 


Turns in homework on time 


62 


76 


3.28 


0.07 


Completes assignments according 
to directions* 


57 


76 


5.88 


0.02 


Pays attention in class* 


54 


74 


6.76 


0.01 


Prepares for class* 


53 


71 


5.07 


0.02 


Prepares for tests* 


50 


67 


4.92 


0.03 


Corrects own work 


50 


60 


1.82 


0.18 


Takes notes in class* 


49 


66 


4.42 


0.04 


Reviews materials* 


41 


59 


4.77 


0.03 



'These differences were statistically significant (p < .05). 



Exhibit reads: According to teachers, 66 percent of students who received free- or reduced-prioe lunch 
“almost always” or “often” completed homework, compared to 78 percent of students who did not receive free- 
or reduced-price lunch. This difference was not statistioally significant. 
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Exhibit A-15 

Teacher Reports of Participants’ Reading and Language Arts Skiiis, 
by Free- or Reduced-Price Lunch Status 



Percent of participants who met at ieast grade-ievel expectations for 
the foiiowing reading and language arts skills 




Received Free- or 
Reduced-Price 
Lunch (n=119) 


Did Not Receive 
Free- or Reduced- 
Price Lunch (n=58) 


Chi-Square 


P 


Oral communication 


71 


83 


3.05 


0.08 


Spelling* 


61 


79 


5.71 


0.02 


Reading fluency 


61 


74 


2.59 


0.11 


Punctuation* 


59 


77 


5.70 


0.02 


Drawing conclusion from written 
material* 


58 


79 


7.79 


0.01 


Grammar* 


58 


79 


7.64 


0.01 


Identifying a main idea* 


58 


78 


6.54 


0.01 


Written communication* 


57 


79 


8.36 


0.00 


Vocabulary* 


56 


79 


9.18 


0.00 


Reading comprehension* 


56 


78 


7.60 


0.01 


Word-attack* 


54 


78 


8.99 


0.00 



'These differences were statistically significant (p < .05). 



Exhibit reads: According to teachers, 71 percent of students who received free- or reduced-prioe lunch met at 
least grade-level expectations on reading comprehension skills, compared to 83 percent of students who did 
not receive free- or reduced-prioe lunch. This difference was not statistically significant. 



Exhibit A-16 

Teacher Reports of Participants’ Technoiogy Skiiis, 
by Free- or Reduced-Price Lunch Status 



Percent of participants who exhibited “excellent” or “good” technology skills 




Received Free- or 
Reduced-Price 
Lunch (n=103) 


Did Not Receive 
Free- or Reduced- 
Price Lunch (n=49) 


Chi-Square 


P 


Playing games 


67 


78 


1.78 


0.18 


Sending and receiving e-mail 


57 


66 


0.60 


0.44 


Use of Internet for research 


56 


65 


0.98 


0.32 


Use of word processing program* 


53 


74 


5.25 


0.02 


Use of spreadsheet program 


39 


42 


0.06 


0.81 



'These differences were statistically significant (p < .05). 

Exhibit reads: According to teachers, 67 percent of students who received free- or reduced-price lunch 
exhibited “excellent” or “good” technology skills, compared to 78 peroent of students who did not receive free- 
or reduced-price lunch. This difference was not statistioally significant. 
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Teacher Survey Scales 



Interpersonal Skills 

Interpersonal skills indicate how well students follow rules, accept limits, and interact with 
adults and their peers. The Interpersonal Skills scale was computed to range from one to five, 
with five indicating that on average participants almost always exhibited the following skills: 

■ Follows classroom rules 

■ Corrects inappropriate behavior when asked 

■ Expresses dissatisfaction appropriately 

■ Accepts suggestions from teachers 

■ Works effectively in a large group activity 

■ Interacts appropriately with adults 

■ Listens to what others have to say 

■ Gets along with people who are different 

■ Works effectively in a small group activity 

■ Interacts appropriately with other students 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.98 


3.92 


0.90 


1 


3.10 


4.90 


5 
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Academic Engagement 

Academic engagement indicates students’ willingness to volunteer to answer questions, ability to 
assume leadership, and readiness to participate in class discussions. The Academic Engagement 
scale was computed to range from one to five, with five indicating that on average participants 
almost always exhibited the following skills: 

■ Speaks in class when called upon 

■ Asks questions about tests or projects 

■ Participates in class discussions 

■ Volunteers answers to questions 

■ Assumes leadership in group situations 

■ Volunteers to read aloud 

■ Initiates conversations appropriately 

■ Asks questions when confused 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.96 


3.66 


0.98 


1 


3.00 


4.50 


5 
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Academic Motivation 



Academic motivation indicates students’ eagerness to learn, willingness to take on challenges, 
ability to stay on target, and responsibility for their own learning. The Academic Motivation 
scale was computed to range from one to five, with five indicating that on average participants 
almost always exhibited the following skills: 

■ Is motivated to learn 

■ Prefers challenging tasks 

■ Produces high-quality work 

■ Critically evaluates own work 

■ Attempts to improve on previous performance 

■ Makes the most of learning experiences 

■ Persists when task is difficult 

■ Looks for ways to academically challenge self 

■ Assumes responsibility for own learning 

■ Is goal-oriented 

■ Stays on task 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.98 


3.24 


1.13 


1 


2.45 


4.00 


5 
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Study Skills 

Study skills illustrate how well students prepare for tests, how often they complete homework, 
and how often they correct their own work. The Study Skills scale was computed to range from 
one to five, with five indicating that on average participants almost always exhibited the 
following skills: 



■ Completes homework 

■ Corrects own work 

■ Finishes class work on time 

■ Prepares for tests 

■ Prepares for class 

■ Turns in homework on time 

■ Takes care of materials 

■ Pays attention in class 

■ Completes assignments according to directions 

■ Takes notes in class 

■ Reviews materials 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.97 


3.69 


0.96 


1 


3.00 


4.55 


5 




A-22 



Reading and Language Arts 

Reading and language arts skills denote students’ ability to identify a main idea, use grammar 
and punctuation correctly, and draw conclusions from written material. The Reading and 
Language Arts Skills scale was computed to range from one to five, with five indicating that on 
average participant reading and language arts skills were far above grade-level expectations for 
the following skills: 

■ Reading comprehension 

■ Word- attack 

■ Vocabulary 

■ Identifying a main idea 

■ Reading fluency 

■ Spelling 

■ Punctuation 

■ Grammar 

■ Written communication 

■ Oral communication 

■ Drawing conclusions from written material 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.99 


2.73 


0.81 


1 


2.09 


3.00 


5 
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Technology Skills 

Technology skills refer to students’ ability to use word processing programs, use the Internet for 
research, and send and receive e-mails. The Technology Skills scale was computed to range 
from one to four, with four indicating that on average participants technology skills were 
excellent for the following indicators: 

■ Use of word processing program 

■ Use of spreadsheet program 

■ Use of Internet for research 

■ Sending and receiving e-mail 

■ Playing games 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.94 


2.74 


0.74 


1 


2.00 


3.00 


4 
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Communication with Parents 



Research suggests that higher parental involvement will yield more positive results in children’s 
academic skills and success. The Communication with Parents scale was computed to range 
from one to four, with four indicating that on average teachers often communicated with parents 
in the following ways: 

■ This student’s parents/guardian attends school-scheduled parent-teachers conferences. 

■ In addition to school- scheduled parent-teacher conferences, I talk in person with this 
student’s parents/guardians about this student’s difficulties and/or progress. 

■ I contact this student’s parents/guardians by phone or e-mail. 

■ This student’s parents/guardians contact me by phone or e-mail. 

■ In addition to scheduled report cards, I send written progress reports to this student’s 
parents/guardians . 

■ I involve this student’s parents/guardians in planning new programming to match this 
student’s needs. 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.85 


3.16 


0.67 


1 


2.67 


3.83 


4 
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By Race/Ethnicity 



Exhibit A-17 

Teacher Reports of Communication with Parents 
by Participants’ Race/Ethnicity 



Communication with Parents 


Pair-Wise Comparison 
(Least Squares Difference Test) 


Race/Ethnicity 


Mean Scale Score 


Comparison 


Sig. 






Hispanic 


0.00* 


Black (n=152) 


3.26 


Asian 


0.05* 


White 


0.07 






Native American 


0.52 






Black 


0.00* 


Hispanic/Latino (n=68) 


2.93 


Native American 


0.06 


Asian 


0.25 






White 


0.73 






Native American 


0.04* 


Asian or Pacific Islander (n=4) 


2.53 


Black 


0.05* 


White 


0.21 






Hispanic 


0.25 






Asian 


0.04* 


Native American or Alaskan Native 


3.40 


Hispanic 


0.06 


(n=10) 


White 


0.12 






Black 


0.52 






Black 


0.07 


White (n=32) 


3.03 


Native American 


0.12 


Asian 


0.21 






Hispanic 


0.73 



‘These differences were statistically significant (p < .05). 



Exhibit reads: On a scale ranging from one to four, with four indioating greatest frequency, the mean 
scale score of teachers’ communication with parents of Black participants was 3.26. The differences 
between teachers’ communication with parents of Black participants compared to Hispanic and Asian 
participants were statistically significant. The differences between teachers’ communication with parents 
of Black participants compared to White and Native-American participants were not statistically 
significant. 
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Details of Data Used in Program Observation Analyses 



In Year 1 of the evaluation of the NJ After 3 initiative, evaluators conducted 10 to 12 
activity observations in each of 10 in-depth study sites. Evaluators used PSA’s Out of School 
Time (OST) Observation Instrument to conduct these structured 15 minute observations. In 
total, observation data represent 179 independent observations and 30 activity co-observations 
with a strong inter-rater reliability coefficient of 0.88. 



Youth Relationship Building 

The Youth Relationship Building scale combines ratings from the following indicators: 

■ Youth are friendly and relaxed with one another 

■ Youth respect one another 

■ Youth show positive affect to staff 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.803 


5.86 


0.81 


3.33 


5.33 


6.33 


7.00 




1.49 1.99 2.49 2.99 3.49 3.99 4.49 4.99 5.49 5.99 6.49 7 

Scale score 
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Staff Relationship Building 

The Staff Relationship Building seale eombines ratings from the following indieators: 

■ Staff use positive behavior management techniques 

■ Staff are equitable and inclusive 

■ Staff show positive affect to youth 

■ Staff attentively listen to and/or observe youth 

■ Staff encourage youth to share their ideas, opinions and concerns 

■ Staff engage personally with youth 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.714 


4.50 


0.87 


2.00 


4.00 


5.00 


7.00 




1.49 1.99 2.49 2.99 3.49 3.99 4.49 4.99 5.49 5.99 6.49 7 

Scale score 
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Instructional Methods 



The Instructional Methods scale combines ratings from the following indicators: 

■ Staff communicate goals, purposes, expectations 

■ Staff verbally recognize youth’s efforts and accomplishments 

■ Staff assist youth without taking control 

■ Staff ask youth to expand upon their answers and ideas 

■ Staff challenge youth to move beyond their current level of competency 

■ Staff employ varied teaching strategies 

■ Staff plan for/ask youth to work together 

Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.807 


3.60 


1.23 


1.00 


2.57 


4.43 


6.71 




Scale score 



A-29 



Activity Content and Structure 

The Activity Content and Structure scale combines ratings from the following indicators: 

■ The activity is well organized 

■ The activity involves the practice/a progression of skills 

■ The activity challenges students intellectually, creatively, and/or physically 

■ The activity requires analytic thinking 



Descriptive Statistics: 



Alpha 


Mean 


Standard 

Deviation 


Minimum 


25* 

Percentile 


75* 

Percentile 


Maximum 


0.829 


4.44 


1.52 


1.00 


3.25 


5.56 


7.00 




Scale score 
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